Polylith Architecture, a prerequisite for digital transformation
Warren Buffet thinks highly of Jeff Bezos, Amazon’s CEO. He has pointed out in Jeff’s praise that the world hasn’t seen many businessmen like him.In his word, overwhelmingly, he’s taken things you and I’ve been buying and has figured out a way to make us happier buying those products, either by fast delivery or prices or whatever it may be, and that’s remarkable,” says Buffet.To achieve this feat, Amazon built an infrastructure and an operations framework to manage it. The credit in making Amazon a true-blue digital customer leader goes to its digital transformation strategy.
Financial service enterprises are currently investing in mainly four areas:
- Infrastructure via “Cloud Strategy”
- Data via “AI and Data Engineering”
- Process via “DevOps”
- People via “Full stack development”
Astechnologists, we aspire to digitally transform our systems but we have to deal with entropy.The second law states that the level of disorder in the universe is steadily increasing. Systems tend to move from ordered behavior to more random behavior i.e.higher the age of system, higher the entropy
We are dealing with systems which are already live and have a varying degree of entropy to deal which impacts digital transformation. At times, we are dealing with monoliths.
What is a monolith
Some may argue that monolith is an architectural concept but simply putit is a lens of pipeline agility. In today’s spacetime of DevOps paradigm, a monolith is defined as following by keeping ‘Pipeline Agility’ in perspective.
|Pipeline Agility||High||Medium||Low (Monolith)|
|Checkout, Build and Unit test||< 15 mins||< 30 mins||> 30 mins|
|Package and Deploy||5 – 10 mins||15-30 mins||> 15 mins|
|Functional Regression Test||< 1 hr||1-2 hrs||>2 hrs|
|Performance Regression Test||< 30 mins||< 1 hr||> 1 hr|
There are many ways to increase your pipeline agility however in situations where basics fail and there is a need to transform a monolith into polylith.
The following 5-step approach has been used for similar initiatives across many organizations:-
- Isolate teams using Conway’s law to develop services
- Consider multiple scalability dimensions (functional and non-functional independence) for isolation. Start splitting from the bottom layer and work towards the top. This helps to identify the dependencies
- Allow teams to warm-up. While the development team that creates the service will get accustomed to the new culture soon enough, the consuming teams will take some time to get used to the “we no longer have control of that piece of functionality”.
- Split by functional attributes (as the Scale-Cubed Model suggest, split by business function),split by quality attributes (where one piece of software requires elevated performance/availability/security etc.), split by technology (where you want to make use of a better tech stack for solving the business problem)
- There should be a strong focus for integration right at the start of the program andall application teams need to be given the mandate to participate in integration workshops.Determine the style of communication (Messaging, RPC, REST etc)
- Agree on Orchestrate (Hub-spoke model) or Choreograph (Smart end-points) approaches
- Test each service with its own robust regression testing pack
- Functional automation test cases are required to ensure success of the engagement. Testing of each iteration and incorporation of feedback is very important to ensure the project is moving in the right direction
- Integration testing is very complex and cumbersome. It also needs several skills to come together. Also, slightest change in end applications calls for quick tests of all integration tests points. Thus, an automated test approach is highly recommended in such projects.
- Deploy separate DevOps pipeline for each micro-service
- It’s a good idea to plan an infrastructure only release where just the infra could be validated before the final release. The new installation should be done on new set of hosts.
- This enables one to do infra only releases.Do not do big bang – Start with a single service implementation.
- Use Agile principals to keep all stakeholders involved and adjust to increased complexity due to introduction of micro service. Introduction of micro services will bring challenges related to training and each micro service may need a lean and dedicated team
- Monitor each service as an independent system
- Centralized logging, health monitoring, smoke testing are extremely critical for maintenance. It is important to plan for operations and support during the design of the solution.
Focus what hurts the most, focus to optimize your pipeline agility and move towards polylith approach.