The Future of Search: Agents, RAG, and Why Retrieval Still Matters — Simon Eskildsen, Turbopuffer
Simon Hørup Eskildsen, founder of turbopuffer, shares his journey from scaling Shopify's infrastructure to creating a new search engine for the AI era. He discusses how a prohibitively expensive experiment at Readwise inspired him to build a cost-effective vector search solution based on object storage and NVMe. Eskildsen breaks down turbopuffer's architecture, its role in cutting costs for companies like Cursor and Notion, his philosophy on building a 'P99' engineering team, and how agentic workloads are changing the future of retrieval.
The Accidental Founding of Turbopuffer
Simon Hørup Eskildsen’s journey to founding turbopuffer began with a simple, haunting insight. After nearly a decade scaling Shopify’s infrastructure, where he developed a deep-seated pain from operating Elasticsearch, he started doing “angel engineering” for various startups. The pivotal moment came at Readwise, shortly after the ChatGPT launch. He built a semantic recommendation engine that showed significant promise but came with an untenable price tag: an estimated $30,000/month, six times the company’s entire infrastructure bill. This stark cost-to-value mismatch for a useful AI feature became an obsession. Eskildsen realized there was huge latent demand for AI search if the cost could be drastically reduced. This led him to do “napkin math” on a new kind of database architecture, which evolved from a side project into turbopuffer.
A New Database Architecture for the AI Era
Eskildsen posits that building a great database company requires three ingredients: a new, ubiquitous workload, a novel storage architecture, and the long-term ambition to support all query plans. He believes we are in such a moment.
- New Workload: Connecting vast amounts of unstructured data to AI models.
- Novel Architecture: Turbopuffer’s core bet is an architecture built entirely on object storage (like S3) and NVMe SSDs, a design that was not feasible until recently. This architecture avoids a traditional consensus layer (like Zookeeper) and is fundamentally stateless, with all data persisting in object storage. This simplicity is a direct response to the pain of managing complex, stateful systems like Elasticsearch.
This approach was enabled by three key cloud advancements:
- The availability of NVMe SSDs in the cloud (c. 2017).
- S3’s shift from eventual to strong consistency (December 2020).
- The introduction of compare-and-swap operations on object storage, which allows for consensus without a dedicated service.
From Scrappy Binary to Powering Notion and Cursor
Turbopuffer’s growth was driven by intense, hands-on work with early customers who validated the architecture.
- The Cursor Story: The day after launching turbopuffer as a minimal Rust binary running in a
tmuxsession, Eskildsen was contacted by Cursor. After a 4 a.m. call, he flew to San Francisco and worked closely with their team. Turbopuffer helped Cursor reduce their costs by 95%, fundamentally fixing their per-user economics. - The Notion Story: This engagement highlighted turbopuffer’s architectural conviction. To meet Notion’s low-latency needs while running on a different cloud (turbopuffer on GCP, Notion on AWS), the team bought dark fiber and meticulously tuned TCP windows to minimize cross-cloud latency, absorbing the cost rather than compromising their architectural principles.
Eskildsen notes that in the age of AI, the “build vs. buy” equation is less about capability and more about time. A dedicated external team that feels like an extension of your own is a massive accelerator.
The Evolving Nature of AI Search
Eskildsen sees search evolving rapidly to serve new AI-driven workloads:
- RAG is Not Dead: The future is hybrid retrieval, combining semantic search with traditional methods like full-text, regex, and structured queries. Both are necessary.
- Search for Agents: The workload is shifting from a single retrieval call to agents firing many parallel queries at once. Search is becoming a highly concurrent tool call, requiring infrastructure that can handle huge bursts of queries.
- Pricing for Agents: To accommodate this shift toward high-volume, concurrent queries, turbopuffer is reducing its query pricing (a 5x reduction was mentioned) to align with agentic usage patterns.
Company Building Philosophy
Eskildsen’s approach to building turbopuffer is guided by a distinct philosophy:
- Radical Honesty: He practices “playing with open cards,” as exemplified by his promise to investor Lachy Groom to return the funds if the company didn’t achieve product-market fit within the year. This authenticity was key to their partnership.
- The “P99 Engineer”: Turbopuffer aims to build a “talent-dense” team by hiring with a “reject by default” mindset. A candidate is only hired if a team member feels strongly enough to fight for them. He looks for traits like obsession and a history of “bending software to their will”—pushing systems to their first-principles performance limits.
The Future of Turbopuffer
The company’s roadmap is laid out in acts:
- Act 1: Vector Search.
- Act 2: High-performance full-text search, optimized for long, LLM-generated queries.
- Future Acts: The focus is on expanding scale (e.g., 100B+ vector search), adding more full-text features, and improving the user dashboard. Long-term possibilities include OLAP queries, logging, and graph capabilities, but the team remains focused on doing things in the right order based on customer needs. The ultimate goal is to be the database that connects AI to vast amounts of data.