Better Search Took This AI Agent From 36% to 98% Accuracy - Aamir Shakir, Mixedbread
Aamir Shakir of Mixedbread argues that the main limit on agent performance is access to the right data, not model intelligence. He explains why OCR, chunking, single-vector embeddings, and vector databases lose information at each step, and presents a late-interaction, multimodal retrieval approach that embeds document pages visually, with a live demo on Intel earnings reports and a hedge fund case study that raised answer accuracy from 36% to 98%.
Why agents fail on documents even when the model is smart
Agents are capable enough for most analytical work. The real bottleneck is getting them to the right information. Take an equity analyst asking why Intel’s newly released GPU uses LPDDR5 memory instead of HBM. The answer is buried somewhere in a knowledge base of broker reports, interviews, podcasts, and research PDFs. The usual approach, which coding agents use on local files, is lexical search: grep-style keyword matching. Keyword hits come back in large numbers with little relevance ranking, so the agent ends up reading nearly every document. Token cost grows with every document added, and a basic question can cost $10 to $20 per query. Scanned pages, images, and binary formats make this worse, because they cannot be searched by keyword at all.
The Frankenstein stack and why it loses information
The standard retrieval-augmented generation pipeline chains transcription or OCR, chunking, an embedding model, a vector database, and reranking. Each stage is lossy. Tables and diagrams that are not parsed properly lose their structure before chunking even starts. Chunk boundaries across figures and multi-page tables are arbitrary. A single embedding then compresses a rich page into one point in vector space. Teams spend six to eight months comparing vendors for each component, clear security reviews, and dedicate four to five people to the build, and quality still degrades as the corpus grows.
The core limitation is that single vectors cannot separate structurally similar documents. A 1996 Nvidia earnings report and a 2026 one produce nearly identical points. A prototype with tens or hundreds of documents looks fine, but production-scale retrieval fails to distinguish the fine-grained figures that matter.
This failure feeds back into agent cost. An agent that cannot find precise information queries aggressively, spins up sub-agents, and fills its context window. Per-query cost then rises with agent traffic, which becomes a serious problem as more fund and VC workflows run through agents.
Intelligence is becoming a commodity, data access is not
The speaker’s central claim is that intelligence is no longer the scarce resource. Models can be swapped as needed. What is scarce is an agent’s access to the right slice of an organization’s knowledge. He expects intelligence and data to unbundle, with teams choosing among models while retaining a dedicated, well-engineered retrieval layer over their own data.
Mixedbread began as a retrieval research lab, motivated by how little the field had moved. Lexical search has its roots in BM25 from 1994, which Elasticsearch still uses at its core. The first strong neural embedding models for text search arrived in 2022, after roughly 27 years. Since then, progress has been incremental and linear rather than the step-function change seen with chat models, which is part of why retrieval-augmented generation never delivered on its promise.
Late-interaction, multimodal retrieval
Mixedbread’s new model uses late interaction. Instead of one embedding per document, it creates many embeddings for each document and for each query, so fine-grained details on a page can be represented separately. Because the document embeddings are computed in a single pass, each one is also contextualized. A net revenue embedding from an Nvidia filing knows it belongs to Nvidia and to a particular year, which resolves the 1996-versus-2026 confusion.
The model also embeds the visual page itself, not only extracted text. A screenshot of a chart or table keeps its layout, so the information survives where OCR would discard it. Mixedbread still exposes layout data and OCR output for downstream systems that prefer text, and bounding boxes let an agent cite exactly where an answer came from, so analysts can trace conclusions back to the source page.
Demo and agentic search
In the demo, Intel earnings reports were uploaded as is, and a query about segment operating margin in the Foundry business for Q2 2025 surfaced the right page with the relevant figures, usually in under 200 milliseconds. For more open-ended questions, such as whether Intel’s Foundry profitability is improving and what drives it, a separate agentic search mode decomposes the query into multiple retrieval hops, gathers evidence, and passes it to an LLM for explanation. The speaker claims this matches leading agentic search systems while being about 40 times faster and cheaper. Plain search, hybrid search, and agentic search are all exposed as primitives, so developers can let the agent choose the appropriate mode for each question’s complexity.
Platform and customer results
The platform ingests PDFs, spreadsheets, images, video, and audio across more than 100 languages, using models trained from scratch for search. It runs on a proprietary database with zero data retention, a bring-your-own-bucket deployment option, and ISO compliance. Banks and hedge funds have onboarded over the last few months, with billions of documents in production.
One large hedge fund with over a billion documents onboarded in about a week and a half. Its previous stack of chunking, OCR, a generic embedding model, and a vector database produced answers that human experts graded correct only 36% of the time. After switching, accuracy reached 98%. Multimodal retrieval accounted for a significant share of the gain on visual documents, and LLM spend fell roughly threefold, because most tokens had previously gone to finding and reading irrelevant text. Better retrieval lets the model work with a smaller, more relevant context.
Teams can benchmark on their own corpora, since large datasets receive substantial free credits, and bring-your-own-bucket setup is activated within about five minutes.