MCP Goes Stateless | ​John Dellenbaugh & Pankaj Kumar | MCP Release Party - Seattle

This session, presented at the MCP Release Party, delves into the significant architectural shift of the MCP (Machine Learning Common Protocol) from a stateful to a stateless design. The speakers, John Dellenbaugh and Pankaj Kumar, illustrate the complexities and limitations of the old protocol through a detailed AI shopping assistant demo and explain how the new stateless approach resolves these challenges.

The Problem with Stateful MCP

John Dellenbaugh opens by recounting a system design interview experience where his proposed horizontally scalable architecture for MCP failed because he was unaware of the protocol’s inherent statefulness, which required a “sticky store” for sessions. This highlighted a fundamental challenge: scaling MCP instances was not as straightforward as typical web services.

Pankaj Kumar elaborates on this problem using an AI shopping assistant scenario. In this hypothetical system, an agent interacts with MCP servers to perform actions like creating a cart, adding items, and retrieving cart information. Under the legacy (stateful) MCP protocol:

• Local Session Management: When an agent initiated a conversation (e.g., create cart) and the request landed on a specific MCP instance (e.g., Legacy A), that instance would create and store the session ID and related data in its local memory.
• Scaling Issues: If the system needed to scale out by adding more MCP instances, or if subsequent requests for the same session (e.g., add item) were routed by a load balancer to a different instance (Legacy B), Legacy B would have no knowledge of the session, leading to “session not found” errors.
• Fragility: The demo showed that while a single instance could handle operations correctly, scaling out immediately broke the system due to lost session context.

Workarounds for Stateful MCP

To mitigate these issues in a stateful environment, two main workarounds were commonly employed:

1. Smart Routing (Sticky Gateway): A dedicated layer (e.g., a sticky load balancer or gateway) was required to maintain a map of session IDs to specific MCP instance IDs. This ensured that all requests belonging to a particular conversation were consistently routed to the same instance. This added infrastructure complexity.
2. Session Store: An external, persistent data store was needed to store session data. This helped in cases where an MCP instance restarted or failed, allowing session data to be retrieved. This introduced another system to manage and maintain.

Both solutions imposed an “infrastructure tax,” adding overhead for managing state rather than contributing directly to product functionality.

The Stateless Solution with Modern MCP

John Dellenbaugh then demonstrates the new (stateless) MCP protocol, which eliminates these complexities:

• Decoupled State: The core change is that MCP instances no longer store session state locally. Instead, when a new session (e.g., create cart) is initiated, a unique identifier (e.g., cart ID) is generated and returned to the agent. This ID is then passed with every subsequent request within that conversation.
• Database as Source of Truth: The actual state (e.g., cart contents) is stored in a backend database (e.g., a PostgreSQL database, illustrated with Koopa Base). MCP instances simply use the provided cart ID to query or update the database.
• Simplified Scaling: With state externalized, standard load balancers (e.g., round-robin) can distribute requests across any MCP instance. The demo shows create cart on Modern B, and subsequent add item requests on Modern A working seamlessly, as the cart ID provides all necessary context for database operations. A large-scale demo with 60 concurrent carts further illustrates horizontal scalability without issue.

Key Takeaways and Considerations

• Security: Statelessness does not inherently provide security. Developers must still implement robust security measures (e.g., protection against session stealing, prompt engineering) independently. John recommends his GitHub repository (JohnDell914) as a resource for learning and extending the demo with security features.
• ACID Principles and Resiliency: The stateless nature of MCP doesn’t automatically solve ACID (Atomicity, Consistency, Isolation, Durability) properties or ensure resiliency. These are still responsibilities of the engineer, requiring application of standard best practices for database transactions, fault tolerance, and data integrity, similar to any other horizontally scaled application.
• Token Management: The specific implementation of passing context (e.g., a cart ID) is application-dependent. The crucial point is that this identifier is stored persistently in the database, not on the MCP server, removing the dependency.
• Real-world Use Cases: An audience member shared potential applications for MCP, including:
  • A prosthetic voice inference CAD modeling agent that uses LLMs (like Grok) to interpret voice commands and call tools to drive CAD software (Unity, Blender, SolidWorks).
  • A carbon fiber espresso machine agent acting as a chatbot to answer complex customer questions about high-end products, leveraging MCP for tool calls to provide detailed product information.
• Growing Ecosystem: MCP is rapidly becoming an industry standard for tool calling, driven by significant investment and a consistent release cadence (e.g., quarterly releases), fostering a robust and expanding community.

• Tags:
• AI Agents
• Stateless Architecture
• Distributed Systems
• Feature
• MCP
• MLOps
• System Design
• Tool Calling
• API Design
• Machine Learning Common Protocol
• Horizontal Scaling