Why We Deleted Our MCP Server and Rebuilt It — Abhi Arya, Reducto
Abhi Arya from Reducto shares how the team went from a one-tool-per-endpoint MCP server that confidently built workflows nobody could explain to a design that makes uncertainty visible, asks instead of guessing, and improves itself through instrumentation. The talk centers on one question: who pays when the agent is wrong?
Abhi Arya, who works on product at Reducto, argues that agent-first software is an architecture problem rather than a model problem, and that it comes down to one question: who pays when the agent is wrong?
Context: Reducto and the production gap
Reducto describes itself as an agentic document platform that turns messy, unstructured documents into data that tools and agents can use. Over the past three years it has processed more than 3 billion documents for customers including Harvey, Scale AI, several of the top five global tech companies, and hedge funds. Abhi’s point is that getting AI to work on a clean document with a frontier model is no longer hard. You can get roughly 60% of the way there quickly. The difficulty is production: scanned pages, rotated pages, and inconsistent layouts, where models start to hallucinate. Reducto positions itself as a layer between the model and the data, handling complex layouts and post-processing so that the agents built on top of it have something reliable to work with.
Three approaches to agent design
Abhi frames the design space as three buckets.
- Auto mode optimizes for autonomy. You write a prompt and let the agent do what it wants. When it fails, it does so with confidence, so you don’t notice, and your customers and users pay for the error.
- User-first optimizes the human interface. You expose every knob, but the agent ends up as little more than a chatbot. This caps what the user understands about the product and what the agent is able to do.
- Agent experience in service of the user is the approach Abhi thinks wins. The agent gets rich but scoped capabilities that do what the user actually wants, and the human can still verify the work. Framed this way, agent quality becomes less a question of the model and more a question of context, capability, and self-improvement.
To illustrate the question, Abhi mentions a Mean Girls-style sticker he received that reads “Get in, loser. We’re building agent-first software.” When his non-AI girlfriend asked him what that meant, he found he had many answers (a good harness, a good API, better prompts), but none of them put the agent first. They just optimize the model.
Discovering the problem: configuration instead of outcomes
Reducto’s pipelines run a document through parsing, classification, splitting, and extraction for use cases like invoicing and contract management. Early on Abhi noticed that users, especially Reducto’s own sales team, spent most of their time configuring pipelines rather than getting outcomes. Legacy interfaces were built for constructing pipelines, while the goal of agentic work is the final deliverable as quickly as possible. The natural move was an MCP server, letting the agent do the heavy lifting so the user only has to describe the use case.
Version one: a tool for every endpoint
Abhi built an MVP with Claude Code, placing every API endpoint behind an MCP tool in one large file. It looked good in a curated Slack demo, which he admits was the wrong test. When the whole team used it, it broke. Someone would paste a half-baked customer call summary or a Notion page, and the agent would build an entire workflow end to end, confidently declaring it correct. Neither the user nor the agent could explain what the workflow actually did. Abhi calls this the root of auto mode in production: the agent never signaled uncertainty, and there was no place for a human to validate its work. The system could talk to the product, but nothing was built to work with it. He deleted the whole file in a single commit.
Fewer tools that carry the structure of real work
The replacement had fewer tools, but each one encoded how the product actually works. For example, routing a classify step into an extract step is a common pattern, so it became a single classify-to-extract tool rather than leaving the model to guess the sequence through prompting. After each step the agent receives contextual guidance about what comes next, and some tools chain automatically.
Because agents are stateless, Reducto also added a snapshot tool that returns the live state of a pipeline: the full flow context and surfaced errors with specific error codes. The agent can reference what is actually there instead of inventing variable names. As Abhi puts it, “you don’t hand your agent an API, you hand it tools that carry the overall expertise such as the right patterns, the state, and the validators so the correct path also becomes the simplest path.” Moving context out of the prompt and into structure is what made the agent stop guessing and start referencing, and it made the output explainable to the humans downstream.
Failures that happen quietly
Fixing building was not the end. The runtime problem showed up when a pipeline ran across 100 documents and quietly missed a scan on page 30, or when the user never specified which document types would flow through. There is no segfault or stack trace in these cases. The agent is confidently wrong and its uncertainty is invisible. Abhi’s response was to make uncertainty legible.
Making uncertainty visible
There are two kinds of uncertainty. The first is outcome uncertainty. Reducto’s Extract already outputs confidence scores and bounding boxes that describe what went wrong, and tools feeding that information back into the model helped optimize extraction schemas over time. The second is the agent’s own uncertainty. When it hits something ambiguous, such as two plausible readings of a field or a document type it hasn’t seen, it should neither guess nor silently stall. It should ask, saying something like “I can read this two ways. What did you mean?”
In their evals, the team started failing agents that built on shaky assumptions, and they rewarded asking. They added prompt guidance that directs the harness to use ask-user-question tools, such as those offered by Claude Code and other frontier harnesses. The agent also explains what it built and why, so the user can review it. Abhi notes that keeping the final judgment in the human’s hands makes one clarifying question do the work of three things: it gives the model context, extends its capability, and keeps a human in the loop.
One root cause
Abhi sees three failure modes that trace back to a single shape. The first was an auto mode nobody could inspect. The second was wrong answers that arrived in silence. The third was an agent that passed evals by reward hacking or hiding what it did. In each case the agent had more autonomy and access than the human could verify. The most valuable and cheapest-to-correct place for the agent’s intelligence is the construction phase, where the human is present. For the parts of the pipeline that cannot be trusted to an agent, Reducto uses its own deterministic document layer to guarantee accurate parses and extractions.
Instrumentation and a self-improving loop
Once the interface was legible, the team instrumented it. Every MCP session logs the prompts sent by the model. Sales and other team members were asked to export their Claude Code sessions. The team tracked where humans overrode the agent, where model confidence was low, and where the agent asked questions. These signals flowed back into the MCP’s long-term guidance. The result was a self-improving loop, per org or per user, in which the system got better at building pipelines simply by building more of them, without manual rewrites.
Adoption without a push
Abhi never told the sales team to use the agent. He exposed it as a product tool in Reducto’s enterprise offering, and the team began wiring it into their workflows on their own. Before customer calls, they researched prospects using CircleBack and Slack, asked the MCP questions about the customer, and built a pipeline for that customer. Because the agent experience was built in service of its users, people reached for it and it became a company-wide resource. They trusted the output enough to show it to high-value customers, with the demo framed as “I built this, come check it out.” Abhi says this paid off in sales calls.
What agent-first really means now
A few months ago Abhi would have said agent-first meant having the best model, the fastest inference, or the most intelligence. With models getting cheaper and more open, he thinks the term now means almost the opposite. The agent does not go first. Building for agent experience means the agent is in the loop with the people who have to answer for the work downstream. Everyone’s agent connects to everything, and Claude can build a new connector on demand, so connectivity is no longer the differentiator. The real question is: when the agent is confidently wrong, who finds out, and how fast? Designing for that is what produces systems people will trust with real work.
Abhi closed by inviting the audience to Reducto’s booth P8, where the company is hiring across several positions.