Can Your AI Analyst Keep a Secret? - Aaron Linsky, Bridgewater & Nikhil Benesch, turbopuffer
Aaron Linsky of Bridgewater Associates and Nikhil Benesch of turbopuffer explain how permission-aware retrieval over a 100-million-document research vault broke a conventional search setup, and why native filtering over a clustered vector index is needed to keep search fast without sacrificing recall.
The permissions problem in private enterprise data
An AI analyst answering a question like why token prices fell last month has to ground its reasoning in facts. Public web documents are visible to everyone, so they can be fed to the model for any user. Internal documents are different: every email, meeting note, and report can carry its own mix of group and individual permissions, so the retrieval layer must return only what the specific person asking is allowed to see. Aaron Linsky frames this as a classic information retrieval problem, and it is the problem turbopuffer was designed to solve.
Bridgewater’s research process and the Vault
Bridgewater has spent roughly fifty years trying to understand how markets and economies work and encoding that understanding as logic it can stress test. The earliest version was a set of if-then rules for trading bonds, written on a yellow legal pad in 1980, which later moved to Excel, Scala, and Python. The current research circle has five steps: perception (seeing what is happening in the world), hypothesis, investigation, systematization, and industrialization, where validated ideas are run and traded every day. For two decades humans sat at the center of that loop, and Bridgewater is now moving machines into the center.
That shift depends on high-quality data. Alongside Delorean, a bi-temporal macro time series database, Bridgewater built the Vault, an unstructured store of about 100 million documents that are indexed, organized, and secured. It draws on 135 external sources, each with its own licensing and access constraints, plus internal writing, meeting recordings, and reports. Investors and agents both read from the Vault, and their outputs flow back in, so the corpus keeps growing and improving.
Modeling permissions and where it broke
Aaron describes a simplified model in which each user belongs to groups and may also hold direct grants on individual documents. A query must combine vector similarity with a filter that checks whether the requesting user can see each match. The first implementation, built without a specialized search engine, seemed to work well. As documents were added to the Vault, however, performance degraded badly, which led the team to evaluate alternative providers.
Why brute force does not scale
Nikhil Benesch starts with napkin math. Assume each document is only 1 kilobyte, roughly a 1,024-dimension embedding at one byte per dimension. Scanning 100 million documents at about 20 GB/s of memory bandwidth takes around 5 seconds, far above the roughly 100 millisecond latency most search workloads need. A list of 10,000 documents is trivial, but scale changes the design. If the data sits on S3 instead of in memory, pulling it into memory can take a minute or more. Brute force is therefore not viable at Bridgewater’s scale.
turbopuffer is built entirely on object storage. It writes directly to S3, GCS, or Azure Blob storage, and it promotes data to SSD or memory only when that data is queried. Cold namespaces cost object storage prices, while hot data performs comparably to traditional disk-based search systems. This architecture lets it scale to terabytes at a fraction of the cost of other systems. It supports keyword, regex, and vector search, and the talk focuses on vectors.
How vector indexes work and why permissions break them
Embeddings place related concepts near each other in a high-dimensional space. Red shoes and maroon sandals sit close together, while a concrete block sits far away. A vector index exploits this by organizing vectors so that a query only needs to examine a nearby region rather than the whole space. Permission filters disrupt this, because the vectors nearest a query may belong to documents the user cannot see, and the index alone cannot account for that.
Post-filtering versus pre-filtering
Post-filtering retrieves the closest candidates and then discards any the user cannot access. It is fast, but for a user with access to only a small slice of the corpus, the search keeps widening until it gives up, perhaps returning 40 results instead of the 100 requested. That hurts recall: a relevant document the user is entitled to can simply never appear, which frustrates users who know it exists.
Pre-filtering inverts the process by building an index of every document the user can access and searching only that set. This preserves recall, but when a user can see half of the vault, the system is back to scanning tens of millions of documents sequentially and loses the benefit of the vector index.
Native filtering
The approach that works combines the two. The vector index automatically clusters similar vectors, currently averaging about 200 documents per cluster. For a given user, the system identifies every cluster that contains at least one accessible document and discards all others without considering them. It then searches the nearest of the remaining clusters and scans only within them. Cluster size is an active tuning parameter, and larger clusters may eventually improve throughput.
Across roughly 400 million vectors derived from 100 million documents, this strategy yields P90 search latency of about 120 milliseconds across all namespaces.
Writing permission filters in practice
Developers express permissions as ordinary filter expressions, such as requiring that a viewers field contains a particular user ID or that a groups field contains any of a set of group IDs. The mapping from an organization’s permission model to these filters is intended to be direct, and turbopuffer handles the complicated query planning underneath. Nested groups remain the hard case, and the team is actively working on making them more automatic.
Beyond permissions
Permission filtering is one part of a broader goal the team describes as isolation-native search. That includes fine-grained access controls within a namespace, regional data residency, keeping data inside a customer’s own VPC, and encryption with keys that the customer’s own customers control. Each of these isolation levels has a corresponding capability in turbopuffer.