Agents for Knowledge Work: Beyond Coding Paradigms

Benjamin Clavié, from Mix Bread, presents a critical perspective on the current development of AI agents, arguing that the industry has mistakenly generalized from the relatively “easy case” of coding agents to the far more complex domain of general knowledge work. He emphasizes the crucial differences and advocates for designing agents that function as true knowledge workers.

The Unique Nature of Code vs. General Knowledge Work

Clavié explains that coding is a highly specialized form of knowledge work with unique characteristics that make it unusually amenable to AI agents:

• Durable Cues: Codebases offer explicit and stable references like identifiers, file paths, and method definitions, which are “literally grepable.”
• Grepable Surface: Keywords and structural elements make code easily parsable and searchable.
• Narrow Tasks: Users typically provide coding agents with highly constrained tasks, such as “close a ticket” or implement a specific feature, rather than expecting them to “discover a new programming paradigm.”

In contrast, general knowledge work (e.g., legal, financial, medical research) presents significant challenges:

• Implicit Meaning: The same string can have vastly different meanings depending on context. For example, “30 days” in a contract could signify a deadline, a grace period, a retention rule, or even refer to a medication schedule in an unrelated domain.
• Contextual and Meaning-Driven: Knowledge requires deep contextual understanding, often lacking explicit function definitions found in code.
• Intent Reconstruction: Agents must reconstruct the user’s intent from open-ended problems, requiring them to break down complex queries and identify relevant information independently.

Defining Knowledge Work

Clavié defines knowledge work in two ways:

1. Input/Output: Main input is information (not physical material), which is often ambiguous and diffuse. The main output is something actionable, such as a judgment, decision, or actionable thought.
2. Topological: If a task requires search, it is a knowledge problem, and if it’s a knowledge problem, it requires search. This encompasses a vast portion of the service economy, including lawyers, academics, actuaries, and software engineers.

Human Solutions: The Tool and Organization Loops

Humans have developed sophisticated approaches to manage knowledge over centuries, which Clavié distills into two intertwined loops:

• The Tool Loop: Traces the evolution of information management from oral tradition to writing, cataloging (Library of Alexandria), bibliographies, and modern search engines.
• The Organization Loop: Shows the progression from individual polymaths to collective knowledge structures like monasteries, universities, bureaucracies, and highly specialized modern firms (e.g., legal practices, hospitals).

These are, in fact, one self-optimizing loop: New knowledge demands better tools, which create new roles and workflows, leading to more efficient knowledge workers who generate even more knowledge, thus perpetuating the cycle.

The Non-Neutrality of Tooling and Practical Impact

Tools are not mere add-ons; they define the scalability and cost-effectiveness of tasks. A task that takes weeks without a tool might take minutes with the right one (e.g., library catalog). Modern multimodal search platforms enable analysis of millions of PDFs, unlocking use cases previously deemed impossible or too expensive.

Clavié illustrates this with benchmark numbers from the BrowseComp+ leaderboard (evaluating search tools on deep research tasks):

• An unoptimized BM25 tool yields ~60% accuracy, rendering it practically useless (“nobody would trust someone that’s right 60% of the time”).
• A well-optimized tool (e.g., a hybrid harness) can achieve 90.2% accuracy while making 20% fewer tool calls. This significant reduction in resource usage (tokens, compute) makes the task economically viable and “worth doing.”

Orchestration: The Law Firm Analogy and Searcher Agents

Real-life knowledge work is rarely a linear, single-query task. Clavié uses the analogy of a law firm breaking down a case:

• A client presents an open-ended problem to a senior partner.
• The partner breaks it down into sub-problems and delegates research to assistants.
• These assistants use various tools, conduct research, and produce concise “memos.”
• The partner then synthesizes these memos to form the final judgment.

This structure highlights the need for orchestration in AI agents. On the MQA benchmark (PDF-based enterprise tasks), Clavié shows that even highly optimized traditional tools (like BM25) and advanced models (like Gemini 3) hit a “tool ceiling,” performing at ~90% accuracy, indicating that the human using the tool also struggles to get higher results with the given tool.

The introduction of multimodal search tools (like Mixbread’s) offers a significant jump in accuracy by allowing agents to process complex PDF data (tables, vision). However, there’s still an “oracle gap” between the agent’s performance (e.g., 88.9%) and human performance (e.g., 99.4%).

To address this, Mixbread introduced “searcher agents”:

• The main agent, facing a complex query, breaks it down into sub-questions.
• Dedicated searcher agents independently research these sub-questions, synthesize findings, and provide short “memos” back to the main agent.
• This architectural change improves accuracy by 3.5 percentage points, reducing the “oracle gap” (the difference between perfect documents and the search system) by roughly 40%.

Key Takeaways for AI Agent Design

Clavié concludes with crucial principles for designing effective knowledge agents:

• Design for Knowledge Work, Not Just Programming: Leverage centuries of human experience in structuring knowledge work.
• Tools Overcome Ceilings: Tools are designed to push past performance limitations. Agents and tools must be co-designed, with agents explicitly aware of and trained to use a diverse set of “primitives” (e.g., grep, BM25, semantic search, multimodal search).
• Orchestration is Critical: Context is a finite resource. Effective agents need to break down complex tasks and orchestrate sub-agents to manage and synthesize knowledge efficiently, as no single context window can hold all necessary information.

• Tags:
• AI Agents
• RAG
• Benchmarks
• Feature
• Multimodal AI
• Information Retrieval
• Organizational Design
• Orchestration
• Machine Learning
• Tool Use
• Knowledge Work
• Retrieval Augmented Generation
• Searcher Agents