Michael Nygard – Architecture Without an End State

Michael Nygard (author of the famous book “Release It!”) gave a talk about the steadily evolving nature of software architecture of a software system (and how to “surf on the wave of change”).

Core problem

• There will be always a three-year plan for an architecture improvement program.
• But faster time to market is needed. The program will never finish because:
  • The CTO gets changed
  • A new technology emerges
  • There is a business event (acquisition, merge etc.)
• So the three-year plan will be restarted every year until you go out of business (because you didn’t finish the architecture improvement program)

• The core problem is: The IT system change slower than the business requirements
  • The best you can do are locally contextual actions!
  • And accept that you’ll get not to an end state!

Recommendations for architectures without an end state:

1. Embrace Plurality
   • There is no single record of truth
   • Epistemological challenges: We can only know what we can record.
   • The model you use to view the world shapes the thoughts you are able to think (reminds me of Wittgenstein: “The limits of my language mean the limits of my world.”)
   • Better approach: Multiple models of data in various systems with the possibility to get additional data (just like hyperlinks)
   • This leads to multiple systems of records
   • Otherwise: Who has the authority of which entries in a global object model?
2. Contextualize Downstream
   • “downstream”: the way from the first backend to the customer’s frontend
   • Downstream systems can add information to the upstream’s data
   • Beware of rich upstream models because there will be many changes downstream if these models are dependent on the upstream model and this model changes
   • Better approach: Multiple objects with specialized attributes with a shared common identifier
   • Enterprise architects need to spend most of their time to determine what the “shared information” are.
     • This includes working out what the representations are going to be used between the various systems
     • But more about the information interchange instead of defining Java versions and middleware platforms!
   • Consumers have to recognize that there is an open world of information
   • Business rules are contextual
     • A system decides if it knows enough to process a request with the given business rules; localize the knowledge
   • Argument data in upstream: You can add fields, but avoid relationships (which are highly contextual and will change often). Otherwise, there will be a large surface area of change.
   • Upstream data are used in business rules that are faced towards the user. These will change more often (whereas rules about augmenting the data change the least often)
   • General rule: Reduce the number of entities a system needs to know about.
3. Beware Grandiosity
   • Avoid enterprise modeling attempts / global object models / enterprise data models
   • Most of these project start but never finish
   • Problems / Challenges
     • You need a global perspective (that is not available because of the contextualized systems)
     • You need agreement across all you business units
     • You need a talent of abstraction
     • You need concrete experience in all contexts
     • And a team that is smart enough, to make local decisions (with the global perspective)
   • Because that’s not possible, you’ll make compromises all the time anyways (which degrades the overall quality of the global object model)
   • A better approach:
     • Seek compromises and subsets everybody can agree on
     • Assume an open world that can be extended anytime
     • Start small with tiny steps to expand the representations
     • Allow lengthy comment periods (with plenty of email reminders because nobody  wants to communicate)
   • Advantages are, that you’re able to explore
     • the (contextual) market space
     • a better/concrete solution space
   • Prerequisites
     • Transparency: Methods, work and results must be visible
     • Isolation: One group’s failure is not able to cause a widespread damage
     • Economics: Distributed economic decision making: Everybody is allowed to make local decisions (=local optimizations) that are in alignment with the global goal
4. Decentralize
   • Misconceptions: “Centralizing IT is a leverage”
     • You’ll get budget fighting instead
5. Isolate failure domains
   • Especially in operations: isolate our system from upstream dependencies
6. Data outlives Applications
   • You’ll have to migrate your data to another system
7. Applications outlive Integration
   • The integrations with other systems will change
   • Build a hexagonal architecture: all systems are connected via explicit boundaries (this makes communication a boundary instead of a domain layer problem)
8. Increase Discoverability
   • Project have many things in common but you don’t know it
   • Do the same things that open source projects do:
     • Internal blogs
     • Open code repositories
     • Modern (source code) search engines
   • “Internal teams should work like open source projects.”
   • Beware: “Budget Culture”
     • Otherwise, contributions to projects are perceived as disruptions
     • Inquiries are alarming because people start to think that there is something wrong with their project
   • Better: “Engineering Culture”:
     • Open sharing
     • Every application should have links to:
       • team blog
       • bug submissions
       • CI server
     • This can be done in a grass root organization

Overall an inspiring talk with great ideas on how to improve software development on an organizational level.

Further information

Podcast about the topic (with focus on resilience): https://devchat.tv/ruby-rogues/217-rr-architecture-without-an-end-state-with-michael-nygard