Building Modern Software at Scale: Architectural Principles • Randy Shoup & Charles Humble
Randy Shoup, SVP of Engineering at Thrive Market, shares insights on architectural evolution, detailing the transition from monoliths to microservices, the principles of building effective platform engineering teams using DORA metrics, and how core distributed systems patterns have both endured and evolved.
On Transitioning from Monolith to Microservices
The vast majority of applications—95-99%—are and should remain monoliths. A transition to a distributed architecture should only happen when you are “dragged kicking and screaming,” not as a starting point. The primary motivations for this transition are:
- Developer Velocity: When a monolithic codebase slows down development because teams are “stepping on each other’s toes,” it’s time to separate components. This is fundamentally an organizational scaling solution that allows teams to work on and deploy their services independently.
- System Load: When specific parts of the system need to scale much faster than others, and simply stamping out more instances of the entire monolith is inefficient or insufficient.
A critical prerequisite for this transition is a deep and stable understanding of the business domain. You cannot effectively split a system into services until the domain decomposition has coalesced. Attempting to do so prematurely leads to solidifying an incorrect architecture, which is significantly harder to refactor than code within a monolith. As a rule, architecture should lag the business; building ahead of business needs leads to wasted effort.
Platform Engineering and Developer Productivity
To improve engineering productivity, you must first ask: “What problem are we trying to solve?” At a company like eBay, the problem was that it took too long to get from an idea to customer value. Using value stream mapping, the key bottleneck was identified in software delivery.
The approach to solving this is to adopt the playbook from the book Accelerate and use the four DORA metrics as a measure of performance:
- Deployment Frequency
- Lead Time for Change
- Change Failure Rate
- Mean Time to Recover (MTTR)
A powerful question to ask development teams is: “If I told you that you had to deploy every day, tell me all the reasons why you can’t.” The answers—flaky tests, manual approvals, long build times—provide the backlog for the platform engineering team. The goal is to implement continuous delivery by automating the entire process, particularly with canary deployments where rollouts and rollbacks happen automatically based on metrics, not manual intervention. At a large scale like eBay, the investment in platform engineering can be as much as 50% of the total engineering organization, justified by the immense leverage it provides to thousands of product engineers.
Building and Running Platform Teams
Platform teams should operate with a vendor-customer mindset, treating their platform as a product and internal teams as their customers. This means prioritizing work based on what provides the most significant impact across the organization.
A key principle is to offer a “paved path” rather than a mandate. The platform should be so effective and easy to use that teams choose to use it because it is strictly better than any alternative they could build, buy, or find in open source. This model creates the right incentives:
- It forces the platform team to stay competitive and relevant.
- It empowers customer teams to select the best tool for their specific job.
Mandating a platform is often a sign of the sunk cost fallacy—“we built it, so you must use it”—which stifles innovation and ignores superior external solutions that may become available.
The Evolution of Distributed Systems Architecture
If building a large-scale system like eBay today, the foundational patterns would remain the same, but the implementation would be radically different.
Enduring Patterns (The “What”):
- Scaling: Domain decomposition and services owning their own data.
- Asynchrony: Event-driven architectures and eventual consistency are essential at scale.
- Automation: Everything from infrastructure provisioning to adaptive configuration must be automated.
- Failure: Designing for failure is critical, using techniques like circuit breakers, graceful degradation, and robust observability.
The Biggest Change (The “How”):
- Cloud & Managed Services: Instead of building private data centers, one would go “all-in” on a public cloud vendor, leveraging their managed services for databases, messaging, and infrastructure. The architecture would be global, multi-region, and auto-scaling from day one.
- Continuous Delivery: The most significant shift over the past 20 years. The industry has moved from bi-weekly, monolithic releases to a continuous stream of small changes. High-performing organizations now deploy multiple times per day per service, a practice that would have been unimaginable two decades ago.