Building a Smarter AI Agent with Neural RAG - Will Bryk, Exa.ai

Will Bryk, CEO of Exa, explains why traditional keyword-based search is insufficient for AI agents and introduces a new paradigm of neural, semantic search. He demonstrates how a hybrid approach, combining neural for discovery and keyword for precision, enables AI agents to perform complex, multi-step information retrieval tasks that were previously impossible.

July 29, 2025 4 min read

Building a Smarter AI Agent with Neural RAG - Will Bryk, Exa.ai — source: https://www.youtube.com/watch?v=xnXqpUW_Kp8

Traditional search engines, born in an era of human users typing simple keywords, are fundamentally misaligned with the needs of modern AI systems. The core insight of early search engines like Google was PageRank, an algorithm designed to rank keyword-matched documents by authority for human consumption. This model, however, fails to grasp the semantic nuance required by Large Language Models (LLMs). A query like “shirts without stripes” often returns shirts with stripes because the system is built on keyword matching, not true understanding.

Rebuilding Search with Transformers

The advent of transformer models like GPT-3, which possess a deep, contextual understanding of language, pointed toward a new foundation for information retrieval. The core idea behind Exa was to apply this technology to search by replacing keyword indexes with powerful document embeddings.

Traditional Approach: Creates an inverted index, mapping keywords to the documents that contain them.
Neural Approach: Uses transformers to generate a rich embedding (a vector of numbers) for every document. This embedding captures not just keywords, but the underlying meaning, ideas, and context.

At inference time, a user’s query is also converted into an embedding. The search process then becomes a matter of finding the document embeddings that are most semantically similar to the query embedding. This allows the search engine to understand complex, paragraph-long queries and deliver results that match the user’s true intent.

Why LLMs Will Always Need Search

While it might seem that self-contained LLMs could replace search, a simple information theory argument proves otherwise. The entire web contains exabytes of data and is constantly being updated. In contrast, a model like GPT-4 stores less than 10 terabytes of information in its weights. Therefore, LLMs will always need an external, real-time source of knowledge. This has led to the now-common RAG (Retrieval-Augmented Generation) pattern, where an LLM queries a search engine to inform its response.

The AIs-vs-Humans Dichotomy

The critical secret is that search engines built for humans are not optimal for AIs. The design principles are entirely different.

“Traditional search engines were built for humans… this slow flesh human that’s typing keywords and wants to read a few links… But AIs are very different.”

AIs require a search engine with fundamentally different capabilities:

Precise, Controllable Information: An AI agent needs a search engine that functions like a database. When it asks for “startups working on something huge that feels like Bell Labs,” it should receive a list of startups, not a set of blog posts that a human is likely to click on. It needs literal, information-dense results.
Search with Deep Context: An AI assistant possesses a rich, multi-paragraph context about its user’s preferences, history, and needs. It should be able to use this entire context as a query to find hyper-personalized results (e.g., apartments, restaurants). Traditional search engines, optimized for a few keywords, cannot handle this.
Comprehensive Knowledge: A human can only process a handful of links. An AI can process thousands of results in seconds. To build a comprehensive report on all companies in a specific space, an AI needs a search engine that can return all relevant results—1,000 or 10,000 of them—not just the top 10.

A Practical Hybrid Search Strategy for Agents

The future of agentic workflows lies in intelligently combining different search modalities. A live-coded demo illustrated this powerful pattern:

Goal: Find the GitHub profiles of engineers in San Francisco who are interested in information retrieval.

This task was broken down into a two-step agentic process:

Neural Search (Discovery): The agent first performs a broad, semantic query: "personal site of engineer in San Francisco who likes information retrieval". This type of discovery-oriented query is where neural search excels, as it can find conceptually related people even if they don’t use those exact keywords.
Keyword Search (Precision): From the list of personal sites, the agent extracts the names of the engineers. For each name, it then performs a highly specific keyword search like "Will Bryk github". This is a lookup-style query where keyword search is more direct and reliable.

This hybrid approach allows an agent to first explore and discover possibilities with neural search and then zero in on specific, known entities with keyword search. An effective search API for AI must provide these different tools, giving the agent the control to decide which one is right for each sub-task. The ultimate vision is a single, powerful API that can handle any query type, turning the entire web into a filterable, queryable database for AI systems.