Building Agent Interfaces: Lessons from Chrome DevTools (MCP) for Agents — Michael Hablich, Google

Agents are a distinct user class with unique cognitive bottlenecks compared to humans. While humans and agents might share the same intent—for example, debugging a webpage—their needs are fundamentally different. Humans rely on visual complexity, layouts, and colors to find signals. Agents, however, operate on tokens and are constrained by context windows. This realization is central to building effective agent interfaces.

From Data Dumps to Semantic Summaries

The initial assumption was that agents, being machines, could handle massive amounts of data. This was proven wrong when the Chrome DevTools team provided agents with raw performance trace files—often 50,000 lines of JSON. This approach consistently overwhelmed the model’s context window, pushing the agent into the “dump zone” where its reasoning capabilities degrade significantly.

The solution was to stop forcing the agent to “read the entire book” and instead point it to the relevant sentence. This was achieved by transforming raw data into concise, semantic summaries in Markdown. Instead of a multi-megabyte JSON file, the agent receives a summary highlighting key performance metrics like Largest Contentful Paint (LCP) and Interaction to Next Paint (INP), enabling it to reason effectively without being overloaded.

Measuring “Tokens per Successful Outcome”

To build better interfaces, a new metric was needed to quantify the “fuel efficiency” of an agent’s interaction with a tool. This led to the concept of tokens per successful outcome.

This metric measures the total token cost (input and output) required for an agent to successfully complete a user journey. The key is “successful outcome”—measuring tokens for failed attempts is pointless, as fuel efficiency is irrelevant if you can’t reach your destination.

A few important caveats:

• Effectiveness First: This metric is a measure of efficiency, which should only be optimized after ensuring the tool is effective (i.e., the agent can complete the task at all).
• Context is Key: This metric should not be compared globally across all tasks. A simple user journey like web scraping will naturally consume fewer tokens than a complex, interactive debugging session. It should be used to compare and improve performance within specific task classes.

This data-informed approach allows teams to identify which user journeys or tools are least efficient and prioritize engineering efforts accordingly.

Designing for Efficiency and Recovery

Reducing token consumption and helping agents recover from errors are critical for building resilient systems.

Three Angles on Token Reduction:

1. Tool Categorization: Niche tools, like those for debugging Chrome extensions, are hidden behind command-line parameters to avoid cluttering the default toolset available to the agent.
2. Slim Mode: This is an aggressive form of categorization that exposes only a few fundamental tools (select_page, navigate_page, evaluate_script). While this drastically shrinks the context window, the trade-off is that the agent may lack the specific tools needed for a task, potentially leading to extra turns or failure.
3. CLI Interface: Providing a command-line interface alongside the MCP server allows the agent to chain commands (e.g., piping the output of one command into grep and then into another command). This offloads post-processing from the model to the local machine, saving a significant number of tokens.

Turning Errors into Recovery Playbooks:

Every time an agent encounters an error, it costs tokens to retry and reason about the failure. Improving error recovery is therefore a direct improvement to efficiency.

• Useful Error Messages: Error messages must be designed for agent self-healing. For example, changing an error from “Unable to navigate back” to “Cannot navigate back, no previous page in history” gives the agent the context it needs to self-correct its plan.
• Proactive Detours: Sometimes a model’s training data leads it to make suboptimal choices. Proactive detours can guide the agent toward a better tool for the job, such as steering it to startPerformanceTrace instead of a generic Lighthouse audit for specific performance debugging tasks.
• Diagnostic Playbooks: For common issues like incorrect setup, pre-packaged skills (e.g., a “troubleshooting” skill) can guide the agent and human user through fixing the problem, enabling self-healing at the system level.

Discoverability: The Schema is the UI

The journey of tool design for DevTools for Agents went through two problematic phases:

1. The Monolith: A single, monolithic debug_webpage tool was too abstract and failed because agents couldn’t compose behaviors.
2. The Overload: Decomposing the monolith into 25 focused tools solved the first problem but created a new one: how could the agent reliably pick the right tool?

The solution lies in recognizing that the tool schema (descriptions, parameters) is the user interface for the agent. Referencing a paper that found 97% of MCP tool descriptions have “quality smells,” the focus shifted to crafting a “minimum viable description.” This is a trade-off, as verbose descriptions consume tokens and can bias smaller models. The key is to:

• Define Purpose: Clearly explain the tool’s core function.
• Provide Usage Guidelines: State the clear activation criteria.

For example, the description for startPerformanceTrace includes keywords for Core Web Vitals (“LCP, INP, CLS”) to help the agent make the semantic connection that this tool is appropriate for page load and performance tasks.

Never Compromise Trust for Convenience

When building powerful tools for agents, security and trust must be paramount. The autoconnect feature in DevTools for Agents allows a human to share their live browser session with an agent for debugging. Users requested a “remember my choice” option to remove the friction of clicking “Allow” on every connection.

In traditional UX, removing this friction would be a clear win. For agents, it represents a significant security risk. The team made a deliberate decision to keep the friction by design. This decision was based on Simon Willison’s concept of the “lethal trifecta” of prompt injection vulnerabilities.

A tiered security model is necessary for browsing agents:

• Tier 1: Local Development: A human is in the loop, granting time-bound access. This is where the consent dialog is essential.
• Tier 2: CI/CD Environments: The agent runs in a controlled, separated environment (e.g., containers, separate Chrome profiles).
• Tier 3: Full Internet Access (“YOLO Mode”): This is the highest-risk tier. It requires strict data separation, domain allow-lists, and robust prompt injection mitigations, as any webpage could be a potential attacker.

A tool like DevTools for Agents might be used across all three tiers, but the security model for each must be radically different. Convenience should never compromise these crucial trust boundaries.

• Tags:
• AI Agents
• Prompt Engineering
• LLM
• Developer Tools
• Security
• Tool Use
• API Design