Omnigent: Composition, Control, and Collaboration for AI Agents
Denny Lee discusses the industry's shift to meta-harnesses like Omnigent, which enables hot-swappable AI models and agents, illustrated by his personal project of using debating agents to plan a matcha farm in Taiwan. He highlights how "tokenomics" is replaying the CapEx-to-OpEx cost shift, emphasizing the need for developer visibility, central governance, and auto-model selection to manage AI spend. The conversation also touches on the importance of databases for agent memory and accountability in AI-assisted workflows.
Denny Lee, PM Director at Databricks and a key contributor to Omnigent, delves into the evolving landscape of AI, highlighting the shift from models to harnesses and meta-harnesses, alongside the re-emergence of tokenomics as a critical cost-management paradigm.
The Agentic Research Workflow: A Matcha Farm Case Study
Lee illustrates the power of AI agents through his personal project: researching optimal regions in Taiwan to grow matcha. Using Omnigent, he employs a “debating agent” workflow, pitting models like ChatGPT against Claude Opus. This process leverages extensive “agentic memory and skills” to establish a comprehensive background.
His research, initially inspired by a deep dive into Taiwanese coffee, focused on Nanto County, traditionally known for tea. Agents helped uncover specific needs for matcha cultivation, such as the necessity for immediate steaming and drying infrastructure post-harvest due to rapid oxidation, a common practice in Japan and Korea but less so in Taiwan. The agents identified Sunga, near Bancha, as the only region in Taiwan with the processing capabilities for oxidized tea. Furthermore, they explored soil pH, elevation, and potential floral notes unique to Nanto County, drawing parallels to coffee beans grown there that surprisingly thrive despite seemingly unsuitable elevation. The core benefit of this debating approach is to “provide me the additional information to say to know what other questions questions to ask,” surfacing insights the user might not have considered.
Forking Conversations and Advanced UX for Agents
Lee identifies a critical missing UX pattern in agent interactions: the ability to “fork conversations.” Omnigent addresses this by allowing users to branch a session, retaining the original shared context while exploring separate threads. This feature proved invaluable in his matcha research, enabling him to explore Japanese and Korean matcha regions in parallel with his primary focus on Nanto County. He laments that typical chat-window interfaces, even CLI, lack this crucial functionality for managing complex, multi-threaded investigations.
Overcoming Language Barriers and Creating New Opportunities
Lee discusses the practical challenges of interacting with agents for real-world scenarios, such as reading handwritten Mandarin from local farmers. He highlights the distinction between Simplified and Traditional Chinese, noting that Traditional Chinese requires more tokens due to its inherent complexity. He mentions research into models specifically for Taiwanese Mandarin and the idea of translating Traditional Chinese to Simplified for processing by LLMs, then back again to save tokens. Despite these complexities, agents allow him to connect with “traditional farmers [and] distributors with the ones that actually have real longevity and real knowledge of the topic,” fostering new opportunities and rediscovering local expertise, challenging the notion that AI solely displaces jobs.
The Meta-Harness Layer: Omnigent’s Architecture
Lee explains that the proliferation of models and harnesses led to engineers managing multiple terminal and browser windows, leading to a loss of context. He draws an analogy to Amazon’s “two-pizza rule” and military span-of-control logic, suggesting that users can effectively manage only five to seven agents before losing track. Omnigent positions itself as a meta-harness, sitting above various models (e.g., Claude Code, Codex, Pi) and other harnesses. Its core value proposition is to enable hot-swappable models and harnesses, allowing users to port configurations, context, and agent memory across different underlying systems. This flexibility is crucial because models and harnesses are constantly evolving. For instance, he describes using Opus for code generation and a specific GPT version for test cases, then later switching to cheaper models like Pi or Sonnet for maintenance, all facilitated seamlessly by Omnigent’s abstraction layer. He likens the meta-harness concept to an “onion” or the Earth’s layers, abstracting complexity to allow users to focus on the problem at hand rather than underlying model parameters.
Tokenomics: The New DevOps for AI
Lee asserts that “tokenomics is the exact same thing” as the CapEx-to-OpEx shift that gave rise to DevOps. Just as developers gained visibility and control over infrastructure spend, they now need to understand and manage token consumption. The challenge lies in distinguishing valuable token spend from wasteful expenditure. Solutions include:
- Developer Visibility: Providing tools like real-time cost trackers to make developers aware of their token spend, influencing more conscientious choices.
- Central Governance: Implementing policies, budgets, and guardrails at the harness layer (e.g., Databricks’ AI Gateway) to control costs and set approval thresholds. This prevents wasteful spending on trivial queries.
- Auto Model Selection: Addressing the developer tendency to default to the most advanced (and expensive) models, Lee suggests using classic machine learning (not another LLM) to intelligently route tasks to cheaper, less powerful models when appropriate. Omnigent is experimenting with this, aiming for a transparent system that notifies users when a model switch occurs and retries with a more capable model if the initial attempt fails. The ultimate goal is vendor-agnostic and efficient resource utilization.
Accountability and the Role of Databases
In the context of AI-assisted development, Lee emphasizes accountability. Referencing debates about “created by” versus “assisted by,” he states that ultimately, “it is still the individual the user that is the one that submitted the code… who’s the neck you’re going to choke.” The responsibility for the outcome, whether code or business projects, lies with the human making the decisions. He also notes that failures often stem from “zero-shot” prompting, highlighting the “garbage in, garbage out” principle.
Finally, Lee points to the “resurgence” of databases, particularly serverless PostgreSQL and Lakehouse solutions, as essential for agents. Agents require “cheap, stateful memory,” which databases provide reliably. This is an “old-fashioned business database problem” dating back to the 1960s, proving that fundamental data management principles remain crucial for building portable, modular, and effective AI agent systems. The ability to abstract, modularize, and port state is paramount for efficiency and success in this new era of AI.