Microservices: The Promised Land of Infinite Scalability

Microservices are one architecture among many that you can employ to build software at scale. Its decentralized approach lends itself well to scale as the needs of your business grow. In this context, scalability doesn't refer to just handling more workload but also, and perhaps more importantly, to the codebase and the engineering organization.

Workload Scalability

When people think of scalability, they most often think about it as the ability to handle more workload. As the number of users increase, and as their use of the app increases, the system needs to handle more workload.

A software system can be scaled vertically or horizontally. Vertical scaling is the strategy of increasing the capacity of the underlying hardware: more CPU cores, faster CPUs, more memory, faster disks. This strategy is inherently capped by the current hardware technology and is rather costly because high-end hardware tends to be quite expensive.

Horizontal scaling is the strategy of distributing the workload among multiple replicas running on cheap hardware. Scalability is achieved by assigning each of the replicas a sliver of the total workload. Horizontal scaling is practically unlimited when the app is run in the cloud. It is also more cost efficient than vertical scaling.

Microservices are not the only architecture that can be scaled horizontally. It is possible for example to replicate a monolithic application behind a load balancer to achieve horizontal scalability. Unlike a monolith however, microservices are independently scalable which means that different microservices can be scaled differently depending on the specific workload that they individually experience. A replicated monolith on the other hand has to be scaled to the maximum workload of any of its components. Independent scalability therefore often means significant cost savings, especially when factoring in the need to scale the underlying data store.

Codebase Scalability

Microservice architectures encourage separation of concerns. Each microservice is an encapsulated module that hides its underlying implementation behind an API, with arguments to and from a microservice passed by value. Under these constraints, it becomes rather difficult for engineers to take the shortcuts that create the tight coupling often seen in a monolith. While a modular monolith enjoys the same benefits, it requires strong discipline to maintain the modularity of a monolith because taking shortcuts is just too easy.

Monoliths often suffer from build-time creep. As more code and more tests are added, the build process time (and cost) gradually increases until it is prohibitively high. Microservices are easier to build, test and validate independently in small chunks.

On a higher level, microservices help contain the chaos of the codebase. Whereas a monolith will have chaos across the entire code base, microservices contain most of the chaos inside each microservice and the rest of it in the interconnections among them. These two tiers make it much easier to understand and reason about a microservice architecture. Chaos contained inside one microservice cannot escape to build upon chaos in other microservices. Chaos at the interconnections of microservices is quite visible and can be more easily spotted and addressed.

Organizational Scalability

Perhaps the most impactful benefit of a microservice architecture is organizational in nature.

Microservices are independently deployable so they can be owned and operated by independent teams and deployed according to their release schedule, allowing high-performing teams to move faster than others.

Microservices employ the principle of information hiding where the implementation of a microservice is not exposed to upstream consumers. Teams can therefore iterated without having to loop in other teams as long as the contract of the microservice's public interface is not violated. This reduces the overhead of cross-team communications which can be very significant in large orgs.

Microservices are small, meaning that they can be owned by small teams that are able to iterate faster because of their reduced communication overhead. Communication complexity is proportional to the square of the size of the team. There are 10 lines of communication in a team of 5, and already 15 in a team of 6. An organizational strategy that splits the org into two-pizza teams (ideally less than 10 people) that own as many microservices as they can handle will often yield the greatest engineering velocity.

Summary

While a microservice architecture is not the only game in town, it is well suited for addressing the technical and organizational scalability challenges of a business as it scales. Without microservices, the complexity of a monolithic codebase often grows to a point where the engineering team can no longer innovate and collaborate efficiently. In most likelihood the entire solution will have to be rewritten at a critical point of the business - when it is growing rapidly - and at prohibitive cost. Investing in microservices from the get-go is a wise investment that mitigates this upside risk.

Buon appetito!