Why Your AI UX Is Broken (and It's Not the Model's Fault) — Mike Christensen, Ably

The standard architecture for AI chat applications, which relies on direct HTTP streaming with Server-Sent Events (SSE), fundamentally limits the quality and richness of user experiences. This model is built around a single client establishing a single, point-to-point connection to a single agent. This tight coupling of the response stream to the client’s connection is the root cause of several significant engineering challenges.

The Limitations of Direct HTTP Streaming

When building AI products, engineering teams often encounter three core issues with the direct streaming model:

1. Lack of Resilient Delivery: The stream’s health is tied directly to the client’s connection. If a user on a mobile device switches from Wi-Fi to 5G, refreshes the page, or navigates away and back, the connection is severed and the in-progress stream is lost.
2. No Continuity Across Surfaces: The connection is a private pipe between one client and the agent. If the user opens a new tab or switches to their phone, they have no visibility of the live, in-progress response from the agent.
3. Limited Live Control: SSE is a one-way communication channel from server to client. This makes it impossible for the client to send signals to the agent while a response is streaming. The only action a client can take is to close the connection, which creates ambiguity for the agent: should it treat the event as a temporary disconnection and buffer events for a potential resume, or as a cancellation to stop generating expensive tokens? This conflict makes “resume” and “cancel” functionality mutually exclusive. Vercel’s AI SDK documentation explicitly states that its abort functionality is incompatible with resumability for this reason.

The Solution: Durable Sessions

A more robust pattern emerging is the decoupling of the agent and client layers using a durable session. This concept introduces a persistent, stateful, shared resource that sits between clients and agents, acting as the medium for all interactions. This approach, often built on a pub/sub model, solves the core problems of direct streaming.

By using a durable session:

• The agent writes events directly to the session, unconcerned with the state of any individual client’s connection.
• Clients connect to the session, not the agent, allowing them to subscribe to activity and resume streams automatically after a disconnection without requiring complex, custom logic on the agent’s side.

Enabling Advanced, Multi-Surface Experiences

The durable session model makes it easier to build the three foundational capabilities that separate a fragile demo from a great AI product:

• Resilience: Clients can reconnect to the session and pick up exactly where they left off.
• Continuity: All clients (across tabs and devices) hold a persistent connection to the session, giving them a synchronized, shared view of all live activity.
• Control: The session provides a bidirectional channel. Any connected client can send messages to the agent, allowing for steering, interruption, or follow-up questions from any device, at any time.

Simplifying Multi-Agent Architectures

This pattern also drastically simplifies multi-agent systems. In a typical model, an orchestrator agent delegates subtasks to specialized agents. With direct streaming, this orchestrator becomes a bottleneck, forced to proxy all granular updates from sub-agents back to the client.

With a durable session layer, all agents can write their progress independently to the session. The client subscribes to this single session to get full, real-time visibility into the work of all agents concurrently, without a complex and fragile proxy architecture.

Ably’s Implementation and Demo

At Ably, this concept is implemented using Ably Channels, which are persistent, addressable, and resumable, providing the necessary foundation for durable sessions. The upcoming Ably AI Transport is an SDK designed to be a drop-in layer that handles the complexity of creating and managing these sessions automatically.

A demonstration of an AI support chat powered by this architecture showcased:

• Automatic multi-tab sync: A conversation stayed perfectly in sync across two browser tabs.
• Flawless recovery: After a forced network disconnection and a page refresh, the stream and session state were seamlessly restored.
• Multi-client control: A task initiated in one tab was successfully canceled from a second tab.
• Concurrent agents: Two different agents (one for product research, one for order cancellation) worked in parallel within the same session, with their activities fully synchronized and visible to the user.
• Human-in-the-loop handoff: A human support agent was able to join the session, instantly see the entire conversation history between the user and the AI, and take over the interaction.

• Tags:
• LLM
• AI
• Feature
• Streaming
• User Experience
• WebSockets
• Real-Time
• Durable Sessions
• SSE
• Ably