Every API Is a Tool for Agents - Matt Carey, Cloudflare

This talk explores how to overcome the context window limitations that prevent AI agents from accessing large APIs. It introduces "Codemode," a technique where agents write code against a typed SDK in a secure, sandboxed environment, moving beyond static tool definitions and enabling full API accessibility.

April 25, 2026 4 min read

Every API Is a Tool for Agents - Matt Carey, Cloudflare — source: https://www.youtube.com/watch?v=YBYUvGOuotE

The primary challenge in connecting AI agents to the real world is enabling them to interact with large, complex APIs. The initial approach to tool-calling, where every API endpoint is defined as a separate tool, fails at scale. For an API like Cloudflare’s, with an OpenAPI spec of 2.3 million tokens, naively converting endpoints into tools would completely overwhelm any agent’s context window.

The Context Window Problem

Initially, the common solution was to fragment APIs into smaller, product-specific MCP (Mega Context Problem) servers. A company might release over a dozen separate MCP services, each containing a handful of tools for a specific product area. While this reduces the context size for any single interaction, it introduces new problems:

Incomplete Coverage: These servers often only include a subset of the available API endpoints, limiting the agent’s capabilities.
User Burden: The user or agent must know which specific MCP server to connect to for a given task, destroying the seamless experience agents promise.

This approach treats the context limit as an MCP problem, but it’s more fundamentally an agent problem. The solution isn’t to shrink the API surface but to evolve how agents discover and utilize tools.

Evolving Beyond Static Tools: Progressive Discovery

The goal is to enable the progressive discovery of capabilities without overwhelming the context window. Several methods have emerged to tackle this.

CLIs (Command-Line Interfaces): Agents can interact with a CLI in a shell environment. By running help commands, the agent can progressively discover commands, subcommands, and parameters. This is effective but has a significant drawback: it requires providing the agent with shell access, which is often a security risk or technically unfeasible.
Tool Search: This method involves a dedicated “search” tool. The agent first calls this tool with the user’s query. The tool performs a keyword search and returns a small, relevant subset of tools (e.g., the top 8 matches) which are then loaded into the context. While this is an improvement, it’s still inefficient, as it often loads more tools than necessary, with only one being used.
Codemode: Letting the Agent Write Code: A more powerful and concise approach is to let the model write code directly against a typed API client or SDK. Instead of providing bulky tool definitions, you provide the agent with type definitions (e.g., from a TypeScript SDK generated from your OpenAPI spec). Types are a very compact way to represent the inputs and outputs of an API. The agent can then generate code on the fly to perform complex actions, chain requests, and handle logic—all within a single “tool” that executes code.

The Security Challenge: Safely Running Untrusted Code

The biggest obstacle to adopting Codemode is the inherent risk of running untrusted, LLM-generated code. This code could attempt to read the file system, exfiltrate secrets, or run infinite loops. This is a solved problem, however, through the use of secure, isolated sandboxes.

Primitives like Cloudflare Workers (specifically, dynamic workers via WorkerD), Deno Sub-processes, and Pydantic’s Monty provide lightweight, secure environments to execute code. These sandboxes offer programmable guardrails, allowing you to:

Disable or restrict network access to approved domains.
Prevent file system access.
Control resource consumption.
Isolate processes so they have no access to host environment variables or secrets.

By executing the agent-generated code within such a sandbox, you can provide full API access without compromising the security of your infrastructure.

The Future of Agent and API Interaction

This shift towards dynamic code generation will reshape both the client (agent) and server (API) sides of the ecosystem.

On the Client-Side:

Programmatic Tool Calling: This will become the default. Agents will have a single, powerful code tool.
Saved “Mini-Scripts”: Users will be able to save successful, agent-generated code snippets for reuse. This creates faster, more reliable actions for common tasks and enables powerful workflows, like a self-healing web-scraping cron job that the agent can automatically fix and redeploy when it breaks.
Stateless Agents: As building clients becomes easier, agents will become more cloud-native, embracing stateless execution loops that are more scalable and resilient.

On the Server-Side:

MCP as Middleware: Exposing your API to agents will become a simple feature flag in your web framework of choice (e.g., mcp: true in a Next.js route).
Lightweight SDKs: The MCP SDKs will become minimal, protocol-level implementations designed for native integration into all major frameworks.

This paradigm allows any developer to expose their entire API surface to agents securely and efficiently, finally fulfilling the promise of giving agents hands to interact with the digital world.