Building Better Software: Why Workflows Beat Code Every Time • Ben Smith & James Beswick • GOTO 2025
Ben Smith and James Beswick explore the evolution of modern software architecture, discussing the critical role of workflow services in managing distributed systems, the trade-offs between microservices and monoliths, and the power of plugin architectures.
The Case for Workflow Services
Workflow services are a powerful tool for managing complexity in modern applications. They offer several key advantages:
- Reduced Code Maintenance: By using visual builders and state machines, developers can orchestrate complex processes with less custom code.
- Serverless and Scalable: Many workflow services operate on a pay-per-use model, scaling automatically with demand without requiring infrastructure management.
- Decoupling: They provide a natural way to decouple services, which is fundamental to building microservices.
Workflows allow developers to visually represent application logic, including branching, loops, and error handling. This makes it possible to implement sophisticated patterns like circuit breakers and retries through configuration rather than writing extensive code. This visual nature is a significant benefit, as it allows any developer to quickly understand the state of a process and debug issues, even months after the initial implementation.
Tackling Distributed Systems Challenges
Building distributed systems introduces challenges that are often abstracted away in monolithic architectures. Workflows provide effective, out-of-the-box solutions for these.
-
The Circuit Breaker Pattern: This pattern is essential for building resilient systems. It prevents an application from repeatedly calling a downstream service that is known to be failing. In a workflow, you can implement this by checking the status of a “circuit.” If it’s healthy, the process continues; if it’s broken (due to a previous error), the workflow diverts to an escalation path, preventing cascading failures and unnecessary costs.
-
Idempotency: Ensuring that an operation produces the same result even if called multiple times with the same input is critical, especially for sensitive operations like payments. A mistake can lead to charging a customer twice or not at all. Workflows can help enforce idempotency by using a unique key for each execution, allowing the system to track whether a specific request has already been processed successfully. It’s crucial to plan for these issues, as they often only surface at scale.
From Monoliths to Microservices (and Back?)
The industry’s journey from monoliths to microservices has been driven by the need to reduce the cognitive load on developers. In a large monolith, developers must understand too much of the system to make changes safely. Microservices, when designed correctly around business domains, allow teams to work more independently.
However, the pendulum may be swinging back slightly. A common pitfall is accidentally creating a “workflow monolith,” where a single workflow becomes overly complex and orchestrates too many disparate services. This recreates the tight coupling that microservices aim to solve. The key is to establish clear service boundaries and use workflows to orchestrate processes within those boundaries.
Decoupling with Event-Driven Architecture (EDA)
EDA is a powerful paradigm for enabling communication between microservices. While a workflow manages the internal state of a single service, events are used to broadcast state changes to other services.
A key design decision in EDA is the granularity of events:
- Thin Events: Contain minimal information, such as just an
orderId. This forces downstream consumers to query the originating service for more details, creating a dependency and potential performance bottleneck. - Fat Events: Contain a rich payload with all the necessary information about the event. This decouples the consumer from the producer’s internal data store but can lead to a different form of coupling via the event contract itself.
The right approach is typically somewhere in the middle, providing enough context in the event to be useful without coupling services too tightly.
Building Extensible Systems with Plugins
Plugin architectures are an underutilized but highly effective way to build applications that can evolve without modifying the core codebase. The “serverless video” application was presented as a prime example. The core application handled video processing and published events at key milestones (e.g., captions_ready). External developers could then build “extensions” that subscribed to these events to add new functionality, such as translating captions into another language.
This model ensures the core application remains stable and decoupled from its extensions. If a plugin fails, it doesn’t bring down the entire system. This approach allows a platform to grow through community contributions without creating a maintenance bottleneck for the core team.
Observability and Developer Experience
The highly distributed nature of modern applications makes them notoriously difficult to debug. This complexity is a primary reason some teams are re-evaluating the “microservices for everything” approach.
While workflow services often provide built-in observability by visually tracking the state of each execution, tracing a single request across multiple services, cloud providers, and third-party APIs remains a significant challenge. This requires specialized observability platforms.
A good developer experience also hinges on the quality of the APIs being used. A great API is characterized by:
- Excellent, clear documentation.
- Feature parity between what can be done in the UI and what can be done via the API.
- Seamless integrations with other services in the developer’s ecosystem.
The Art of Developer Advocacy
The role of a developer advocate requires a unique combination of skills:
- Technical Credibility: A deep understanding of the technology.
- Empathy: The ability to understand and relate to the user’s problems.
- Communication: The skill to convey complex ideas clearly across various mediums (writing, video, public speaking).
- Independence: The drive to work autonomously and manage one’s own projects.
A great developer advocate is willing to be opinionated, not to declare a single “right” way, but to explain the reasoning and trade-offs behind their decisions, providing a clear path for others to learn from.
Advice for New Developers
Two key pieces of advice were offered for those new to the software industry:
- Share Your Learning: As you learn something new, write about it, create a video, or share code on GitHub. This not only solidifies your own understanding but also connects you with a community of peers facing the same challenges.
- Embrace Experimentation: Don’t be afraid to get things wrong. Software development is a series of trade-offs, and true learning comes from making decisions and understanding their consequences. Every mistake is an insightful lesson.