Scale, Flow & Microservices • James Lewis • YOW! 2019

James Lewis from Thoughtworks explores the surprising connections between complex adaptive systems, organizational design, and software architecture, suggesting we may have found the underlying principles for creating high-performing, scalable organizations. The core idea is that the same scaling laws that govern the growth of mammals and cities can explain why companies slow down and how certain architectural patterns, like microservices, can reverse this trend.

Corporate Metabolism and The Problem of Scale

It’s a common experience that as organizations grow, their “corporate metabolism” slows down. Adding more people often makes things slower, not faster, a concept famously captured in the Mythical Man-Month. However, some companies, like Amazon, defy this trend. As they get bigger, it becomes easier for them to build more products and add more teams.

This phenomenon can be explained by scaling laws, a concept from complexity science.

• Sublinear Scaling in Biology: Mammals exhibit sublinear scaling. As a mammal doubles in size, it only needs about 75% more calories to survive, not double. This efficiency comes at a cost: the metabolic rate of cells in larger animals is slower. An elephant’s cells operate more slowly than a mouse’s, leading to greater longevity but a slower pace of life. This is a direct consequence of its deep, hierarchical circulatory system.
• Sublinear Scaling in Companies: Similarly, as companies double in size, they typically see only an 85% increase in revenue for an 85% increase in infrastructure costs. They become more efficient (economies of scale) but also slower. This slowdown is caused by the deepening organizational hierarchies required to manage the increased size.

The Physics of Organizations: Complex Adaptive Systems

The Santa Fe Institute defines Complex Adaptive Systems (CAS) with three core principles that apply to everything from life to cities to companies:

1. Space-filling fractal networks: These are the distribution systems, like a circulatory system or an organizational chart.
2. Invariant terminating units: These are the fundamental components, like cells in a body or employees in a company.
3. Optimization through feedback: This is the process of improvement over time, such as evolution or market forces.

Hierarchical networks, which are common in corporate structures, inevitably lead to sublinear scaling. The deeper the hierarchy, the slower the flow of information and work, much like blood flows more slowly to the extremities of a larger animal. Queues in our value streams—long backlogs, approval gates—act as “blockages in corporate arteries,” deliberately slowing the system down and starving the “invariant terminating developers” of the work they need.

The Alchemist’s Stone: Superlinear Scaling

While hierarchical systems lead to slowdowns and eventual corporate mortality (the half-life of a public company is only 10.5 years), there is another model: superlinear scaling.

Cities are the prime example. They exhibit both:

• Sublinear Scaling (Economies of Scale): Infrastructure like roads and power lines scales at the ~0.85 rate. Doubling a city’s size requires less than double the infrastructure.
• Superlinear Scaling (Returns to Scale): Socioeconomic outputs like patents, wealth, and innovation scale at a ~1.15 rate. Doubling a city’s size produces more than double the innovation.

The reason for this is that cities are not just hierarchies; they are also dense social networks. These networks allow for random, cross-disciplinary interactions that spark innovation. You get to choose who you interact with, creating a fertile ground for new ideas.

Amazon, Microservices, and Networked Organizations

Amazon’s success can be attributed to deliberately designing its organization to function like a city. Jeff Bezos, a patron of the Santa Fe Institute, imposed a set of constraints with his famous 2002 memo:

• All teams must communicate via service interfaces.
• All interfaces must be designed to be externalizable.
• Teams must be small (“two-pizza teams,” aligning with Dunbar’s number for social groups).

These rules forced the creation of a network of decoupled, independent teams rather than a monolithic hierarchy. This fostered social network dynamics over hierarchical ones, leading to the observed superlinear growth in revenue (~1.15), where they make more money than their size would predict.

The microservices architecture movement, in essence, stumbled upon the same principles. Characteristics like organizing around business capabilities, decentralization, and smart endpoints with dumb pipes are all methods for creating these small-world networks within an organization. We are trying to build systems that grow on the edges, like a city, rather than slowing down from the top, like an elephant.

Practical Takeaways

1. Track Organizational Health: Use the four key metrics from the book Accelerate (Lead Time, Deployment Frequency, MTTR, Change Fail Rate) as leading indicators of your organization’s metabolic health. A Change Request Board, for example, is a predictor of low performance because it’s a deliberate blockage in the artery.
2. Recognize the Limit: A purely hierarchical organization has a physical limit on its performance, dictated by its structure.
3. Experiment with Social Networks: Foster innovation by creating small, independent, product-focused teams that can operate as a network, reducing the constraints of the hierarchy.
4. Think Externalities: Design services as if you might sell them one day. This forces clear interfaces and true decoupling, as demonstrated by the success of AWS.

Ultimately, the structure of our organizations and architectures dictates their performance. By shifting from rigid hierarchies to dynamic networks, we can build systems that, like cities, become more innovative and resilient as they scale.

• Tags:
• Feature
• Scaling Laws
• Complex Adaptive Systems
• Organizational Design
• Microservices Architecture
• Conway's Law
• High-Performing Teams
• Amazon
• Santa Fe Institute
• Thoughtworks