Version-Controlled Search: The Web as It Was — Ryan Ahern, Exa
Ryan Ahern of Exa presents Exa Snapshot, a point-in-time web search that lets agents query the web as it appeared on any past date, and shows how teams can backtest predictive signals and financial models without overfitting.
Agents are becoming the primary searchers
Ryan Ahern, a forward deployed engineer at Exa, frames the talk around two ideas: why agentic web search matters, and how point-in-time search can be used to backtest financial models. He opens by asking the audience to trace their own search habits, from typing into Google to asking ChatGPT, Claude or similar agents to search on their behalf. His core claim is that agents increasingly do the searching for people, and they search very differently from humans. A human typically tries a few keywords, checks the first results and gives up. An agent writes long, detailed queries, retries when results are weak, and runs many searches per task before it answers. Each task therefore generates far more search calls than a human session would, which makes the quality of the search backend matter much more.
Exa positions itself as that backend. Tools such as Cursor fetching the latest documentation for a coding agent, or Gamma pulling Apple SEC data from 2018 into a slide deck, call Exa under the hood. Ryan notes that Exa runs its own crawling infrastructure, database, embedding model and retrieval architecture, which lets it tune the whole stack for agents. He highlights financial customers in particular: hedge funds looking for alpha, insurers assessing counterparties and risk, sovereign wealth funds, asset managers, and VCs hunting for startups before they appear in any structured dataset.
Exa’s three principles: quality, cost and latency
The first principle is quality. Better retrieval means fewer tool calls, which lowers inference cost and raises the chance that an agent completes its trajectory without being misled. Ryan mentions a recent benchmark designed to evaluate search tools separately from the model calling them, since the two are easily confounded.
Cost is the second principle. He argues that what matters is not the price of a single tool call but the total cost of the agent finishing its task. Exa’s dynamic highlights extract the exact relevant snippets from every result, rather than returning full pages or highlights from only one page. Ryan says this delivers higher-quality context at about 95% fewer tokens than full-page text, with corresponding savings on downstream input tokens.
Latency is the third. Owning the infrastructure allows very low-latency search. At the other end of the spectrum, Exa Agent Ultra targets long-running deep research, which Ryan describes as state of the art on several benchmarks. Because Exa controls both the search harness and how it interacts with the agent, it can tune behavior for lengthy research tasks.
Exa Snapshot: searching the web as it was
The main announcement is Exa Snapshot, a collection of more than 400 billion web snapshots that can be searched by date. It builds on freshness checks Exa already performs: each time its crawler revisits a URL to confirm the content is current, it can store a version, producing a version history. Two access patterns follow. The first retrieves a URL as it appeared on or before a given date. The second, more consequential, is version search, which returns the results a query would have produced at a past point in time, including only pages that existed then.
Ryan illustrates this with OpenAI’s revenue. Snapshot results from early October showed reporting of an annualized run rate around $70 billion, which shaped how the market projected the company’s valuation. Later publications, following earlier Axios reporting, revised the figure to roughly $50 billion on a net basis, after accounting for revenue passed through cloud providers. The point is less the accounting than the information landscape. One can see what the public web said before market consensus formed, including SEC footnotes, lesser-known vendor reports and analyst commentary, and evaluate which sources were credible at the time.
From signals to predictions: backtesting without leakage
The central argument is that live search contaminates backtests. A live search returns everything, including the eventual outcome. Ryan’s example is predicting whether a company will be acquired: a live search surfaces the announced deal, so any model trained on it effectively sees the answer. Snapshot lets the practitioner query as of one day, seven days or one month before the announcement, and then check whether the model would have predicted the acquisition across a set of companies.
He outlines three archetypes of how teams currently use this approach:
- Traditional models, possibly decision trees or simple ML, that enrich structured data with unstructured web signals such as software pricing, sentiment, or the terms and conditions of a site. Historical versions of such pages are often not reliably available any other way.
- Existing models augmented with agentic features, such as an adverse media scan that runs multiple searches and condenses the results into structured fields, or equity research that gathers revenue or headcount estimates across a universe of companies. Teams can ask what such a scan would have returned a month earlier, or four months before onboarding a position into a portfolio, and compare that against outcomes.
- Forward-looking systems in which an agent manages the full prediction process and calls the web as one tool among many. Here snapshot search acts as a control. Rather than building an internal corpus that imitates the web, teams can ensure future information does not leak into the model’s view.
Testing generalization and avoiding overfitting
Ryan describes an extended workshop with about 100 participants who built features for predicting acquisitions. Examples included whether an investor or significant shareholder publicly called for a sale of the whole company within the past year, whether the share price had fallen at least 60% below its five-year high, and, as a deliberately bad example, whether the company was named Dominion. He presents the last one as classic overfitting that a proper backtest should expose.
The broader point is that snapshot makes it possible to test whether a feature generalizes rather than merely memorizes the training data. Without historical search, teams might wait months in production to learn whether an agentic pipeline is predictive or overfitting. With it, they can assemble a labeled set of cases, perhaps around fifty examples, hold some out, and check performance against the holdout. Feature descriptions can be arbitrarily complex, for instance adverse media against VCs who appeared on the AI Engineer podcast and made negative statements about a company. Semantic search can handle such queries, and each one can then be backtested to see whether it actually performs.
Takeaways
Ryan closes with two recommendations. First, evaluate whether Exa’s search, which he frames as high quality, cost-effective and fast across latency profiles, adds value to your agents. Second, use Exa Snapshot to backtest financial or risk models that incorporate web-derived features, so you can determine whether those features improve performance before relying on them.