Decagon’s Playbook for Building Enterprise AI Applications
Jesse Zhang and Ashwin Sreenivas, co-founders of Decagon, discuss their company's transition to open-source models for enterprise AI, emphasizing how fine-tuned small models outperform frontier models on specific tasks. They delve into the role of application-layer companies in an AI-first world, their product-driven 'glass box' approach for enterprises, and the transformative power of their 'Duet Autopilot' agent, which builds other AI agents. The conversation also covers AI's impact on jobs, highlighting the Jevons Paradox in customer support.
Decagon, led by co-founders Jesse Zhang and Ashwin Sreenivas, has transitioned from initial reliance on frontier AI models (like OpenAI and Anthropic) to predominantly using open-source models for 90% of its operations. This strategic shift was driven by critical needs for lower latency, essential for their voice agents, and to effectively scale with large enterprise clients. A core finding for Decagon is that smaller, fine-tuned open-source models can outperform larger, general-purpose state-of-the-art models on specific tasks. This approach defies the perceived trade-off between intelligence and efficiency, as these specialized models deliver superior performance, reduced cost, and faster inference for targeted applications. The remaining 10% of Decagon’s workflow still utilizes closed-source frontier models for more exploratory or broad tasks, such as their innovative “Duet Autopilot” agent, which requires extensive reasoning capabilities to analyze millions of conversations, identify trends, and iteratively improve primary AI models.
Decagon Labs functions as a sophisticated “model factory,” dedicated to continuous training and adaptation of models in response to the rapidly evolving AI landscape. This necessitates a substantial internal investment in a specialized research team focused on fine-tuning, data management, and developing bespoke evaluation benchmarks that are directly tied to tangible customer outcomes, rather than relying on generic performance metrics. For Decagon, performance, latency, and accuracy are the primary drivers, with cost optimization serving as a beneficial side effect. They emphasize that for a growth-stage company, the value delivered per unit of output (e.g., a customer conversation) is far more important to clients than the underlying token costs. While enterprises are expected to eventually adopt open-source models for scaled production, this transition is anticipated to be gradual due to the significant internal capabilities required for effective fine-tuning, data handling, and model validation.
The speakers actively challenge the notion that frontier AI labs will be the ultimate disruptors, arguing for the enduring importance of application-layer companies. They contend that even with the advent of Artificial General Intelligence (AGI), AI agents will still depend on applications for fundamental functions like data storage, information retrieval, and complex reasoning processes. Applications provide an indispensable layer for embedding intricate business logic, managing integrations, facilitating testing, ensuring quality assurance, and maintaining compliance—tasks that are beyond the inherent capabilities of foundational models. Decagon’s perspective is that while frontier models offer broad intelligence, core industry verticals demand deeply specialized solutions underpinned by robust platforms that integrate specific business logic and workflow management. This creates a convergent environment where frontier labs expand into application-like features, and application companies develop highly specialized models for their respective domains, ensuring the longevity of the application layer.
Decagon’s interpretation of “forward-deployed engineering” (FDE) distinguishes itself from mere consulting. While FDEs are crucial in the nascent stages of AI adoption to decipher novel workflows with customers, their ultimate mandate is to “eat pain and excrete product.” This means their direct customer engagement and insights are funneled back into developing scalable core product features that benefit a multitude of clients, rather than creating bespoke, one-off solutions. This product-driven philosophy is a key differentiator, empowering clients with control and self-service capabilities for rapid iteration, contrasting sharply with a “black box” approach often associated with competitors like Sierra. Decagon’s “glass box” model, prioritizing customer autonomy and swift iteration, has been instrumental in securing large enterprise contracts.
The “Duet Autopilot” agent serves as a prime example of this productization strategy. Initially, the process of building conversational AI agents involved extensive manual tasks, such as authoring “Agent Operating Procedures” (AOPs), creating integrations and tools, developing tests, and monitoring conversations. Duet Autopilot, conceptualized as a larger, slower “second agent,” now automates these complex tasks. It can analyze transcripts and documentation, infer optimal procedures, generate tests, simulate scenarios, proactively identify areas for improvement, and even draft necessary changes. This capability highlights a significant advancement in AI—the ability for AI to build, manage, and refine other AI systems.
Decagon’s evolution from focusing solely on customer support to embracing an “AI concierge” vision reflects the continuous improvement in model capabilities, particularly in their ability to follow complex instructions and apply common-sense reasoning. The initial emphasis on customer support was a direct response to market demand and the capabilities of models at the time. As AI technology advanced, Decagon expanded its offerings to include sales qualifications and sophisticated operational workflows, framing these as AI agents executing business processes. Their long-term vision positions the AI agent as the primary “front door” of any business, handling all customer interactions—both reactive and proactive. The product roadmap, rather than being rigidly fixed, dynamically evolves, with strong customer signals driving development priorities.
Regarding operational bottlenecks, Decagon identifies talent acquisition as its primary challenge, rather than inherent technological limitations of AI models. Despite being “voracious consumers of tokens,” they continuously seek skilled individuals, recognizing that model improvements actually increase the potential for innovation, thereby amplifying the need for human talent to build more. They note that AI agents are not yet capable of strategic decision-making, such as determining “what to build” or “what to exclude.” Culturally, Decagon fosters an environment of ambitious individuals working diligently towards shared objectives, promoting a “team sport” ethos with strong inter-organizational collaboration, reinforced by an in-office culture and structured international onboarding programs.
Finally, concerning the societal impact of AI on employment, Decagon believes AI will “kill jobs but not careers.” They observe the Jevons Paradox playing out in customer support: by making support operations more efficient and cost-effective through AI, companies are prompted to provide more support, uncovering previously unmet customer needs and reallocating human roles to more valuable, revenue-generating, and complex tasks. Mundane and repetitive jobs are increasingly automated by AI, thereby freeing human workers to engage in more meaningful, creative, and higher-level responsibilities, fostering career evolution rather than widespread job displacement. They also foresee CRM systems continuing to be essential as data repositories for AI agents, rather than being made obsolete.