Bending a Public MCP Server Without Breaking It — Nimrod Hauser, Baz
Learn practical strategies to adapt third-party MCP server tools for production AI applications. This talk covers five key practices: curating tools, enhancing descriptions, implementing deterministic guardrails, composing new tools from existing ones, and leveraging tools as simple functions, all demonstrated through a real-world "Spec Reviewer" example.
Nimrod Hauser’s talk addresses the critical challenges of integrating third-party Managed Code Provider (MCP) servers and their tools into production AI systems, particularly for agentic workflows. While these tools appear ready-to-use, they often lead to unpredictable agent behavior, performance degradation, and significant security risks, especially in multi-tenant architectures where agents might lack architectural awareness.
The Problem: “Bending a Public MCP Server Without Breaking It”
The core issue stems from the generic nature of third-party tools, exemplified by Playwright’s MCP server in the presented use case. Tools are essentially callable functions wrapped with descriptions that guide agents. However, these descriptions are often too shallow for specific use cases. Initial attempts to use tools “vanilla” (e.g., using LangChain’s load_mcp_tools directly) result in:
- Agent Hallucination: Agents creating non-existent paths or misinterpreting tasks.
- Subpar Performance: Inefficient actions, incorrect results, or failures to complete tasks.
- Security Vulnerabilities: Lack of guardrails leading to potential data leaks or unauthorized actions.
In a toy example of Baz’s “Spec Reviewer” (an agent that compares requirements with implementation, using Playwright to interact with systems), an initial run failed to navigate correctly, resulted in a 404 error, and couldn’t take a proper screenshot, yielding a “failed verdict.”
The Solution: A Five-Step Framework for Tool Optimization
Hauser introduces a framework of five best practices to transform brittle setups into production-ready infrastructure, focusing on both context engineering and deterministic guardrails.
1. Curating Third-Party Tools (V1: Curated)
The first step involves excluding unnecessary tools from the agent’s context. Public MCP servers provide a wide array of tools (e.g., Playwright offers 21). Many of these may not be relevant or even desirable for a specific application. By filtering out unneeded tools (e.g., browser_resize, drag_and_drop, run_code), the context window for the agent is reduced, simplifying decision-making and potentially improving performance. In the example, the tool count was reduced from 21 to 16.
2. Wrapping Third-Party Tools with Enhanced Descriptions (V2: Wrapped)
Generic tool descriptions provided by MCP servers are insufficient for guiding agents precisely. This practice involves creating custom, enhanced descriptions tailored to the application’s specific needs. For example, for a tool like browser_click, the description can be augmented with specific guidance: “First call the accessibility_snapshot tool, and then call this one.” This allows developers to inject their experience and preferences, influencing the agent’s behavior and making it “more eager to choose one tool over the other.” The actual tool functionality remains the same; only its description is updated, ensuring the agent interacts with it as intended.
3. Adding Deterministic Guardrails (V3: Guardrails)
For sensitive or mission-critical aspects, agents’ non-deterministic nature can be a liability. This practice introduces deterministic logic to enforce specific behaviors and prevent agents from going “rogue.” A concrete example involves the take_screenshot tool: to prevent agents from saving images to unauthorized locations, a guardrail was implemented to validate the target path. If an agent attempts to save outside a designated screenshots_root folder, the tool’s invocation is halted, and an agent-facing explanation is returned (e.g., “Access is denied. You need to save it here. Please provide a proper file name and proper path. “). This aligns the agent, prompting it to correct its action rather than crashing the process.
4. Composing New Tools from Existing Tools (V4: Composed)
Existing tools can serve as building blocks for more specialized tools. By leveraging the functionality of a base tool (e.g., take_screenshot), a new tool can be created with a distinct description and potentially additional pre/post-invocation logic. For the Spec Reviewer, an evidence_tool was composed from the take_screenshot tool. This new tool has a specific description guiding the agent to use it “specifically for the purposes of evidence” and instructing it to include the ticket number in the saved file name. This allows the agent to differentiate between general screenshot needs and evidence collection, leading to more structured and context-aware actions.
5. Treating Tools as Simple Functions (V5: Deterministic Functions)
Sometimes, it’s more effective to call third-party tool functionalities directly as deterministic functions outside the agentic flow. This is particularly useful for complex, repetitive, or sensitive steps that should always happen in a specific way, unburdening the agent from unnecessary cognitive load and potential errors. The login_to_Buzz function in the example is a prime case. Logging in can involve complex, varied mechanisms (e.g., JWT token injection into browser local storage) and is a prerequisite for every Spec Reviewer task. By handling this deterministically, the agent receives control after a successful login, allowing it to focus on its core task without navigating the intricacies of authentication.
Conclusion
By iteratively applying these practices, the initial “busted MCP server” was transformed into a “beautiful, working” system. The final run of the Spec Reviewer successfully identified the configuration drawer, provided a correct “passed” verdict, and saved an appropriately named screenshot as evidence. The key takeaway is that integrating third-party agentic tools requires active tailoring and a flexible approach, balancing deterministic control with agentic flexibility to achieve optimal, secure, and reliable production-ready AI applications.