Lessons from Trillion Token Deployments at Fortune 500s — Alessandro Cappelli, Adaptive ML
95% of GenAI pilots fail due to feedback integration issues, not deployment challenges. Alessandro Cappelli argues that Reinforcement Learning (RL) provides the only systematic way to incorporate business metrics and production signals to continuously improve models, especially for complex agent-based systems.
The prevalent belief that moving a Generative AI model from a pilot to production is merely a “last mile” problem is a myth. The reality is that getting to a minimum viable product (MVP) is just the first step; the real marathon is the journey from MVP to production and the continuous improvement required beyond it. The staggering 95% failure rate of GenAI pilots is not a deployment or prompt engineering issue, but fundamentally a feedback integration problem.
The Limitations of Conventional Approaches
Most MVPs are built using proprietary models or open-source models with instruction fine-tuning (SFT). Both methods lack a systematic, mathematical way to incorporate feedback and defects discovered during testing and production.
- Proprietary Models: The primary method for improvement is adjusting system prompts. This is not a scientific process; a change to fix one defect can often introduce others, making consistent, predictable improvement difficult.
- Instruction Fine-Tuning (SFT): The only recourse is to iteratively rebuild the dataset, which is expensive and impractical for continuous improvement post-production. Asking “Will you keep creating a new data set every single week?” highlights the unsustainability of this approach.
The secret to successfully navigating the journey to production is accelerating the model lifecycle through the continuous integration of feedback from clients, business metrics, and production signals. Reinforcement Learning (RL) is uniquely designed to unlock this capability.
The Outsized Advantage of Reinforcement Learning
While prompting, SFT, and RL all aim to steer a model’s behavior, they are not equally effective. RL is disproportionately more effective, delivering outsized performance gains. This means you can achieve the same level of performance with a much smaller model compared to one trained with SFT. This efficiency unlocks three critical advantages for production-grade systems:
- Favorable Tokenomics: Smaller models are significantly cheaper to serve at scale. For large enterprises like AT&T, which summarizes every customer-agent transcript, this can translate into millions of dollars in savings and make a use case economically viable.
- Low Latency: Smaller models are faster. Many use cases, such as real-time speech-to-speech customer support, have strict latency constraints (e.g., under a third of a second) that are impossible to meet with large models but achievable with smaller, RL-tuned models.
- Ownership and Stability: By training your own specialized model, you own the solution and the data it’s trained on. This insulates you from unexpected performance shifts caused by underlying updates to proprietary models.
RL: The Native Framework for AI Agents
The advantages of RL become even more pronounced in the era of AI agents. Agents introduce greater complexity, require more tokens, and have a much lower tolerance for error, as they can directly access and modify live databases. RL was fundamentally designed to train agents to operate within an environment, making it the natural and superior choice.
- Environments and Synthetic Data: If a production environment for an agent already exists, an RL model can be plugged in directly. If not, a mock environment can be built with mocked tools and users. A crucial byproduct of this setup is a synthetic data generation pipeline. As the model explores the environment, the reward signal identifies good and bad trajectories, which can be used to create a high-quality dataset to bootstrap model training—solving the common problem of non-existent agent training data.
- Leveraging Existing Data: Enterprise data, like real customer transcripts, can be used to make mock users in the environment more realistic, leading to more robust and effective models.
Redefining the “Human in the Loop”
Reinforcement Learning from Human Feedback (RLHF) is often associated with expensive, time-consuming annotation campaigns. A more efficient paradigm shifts the human’s role from annotator to architect of the reward signal. The human defines the rubrics, and the reward signal is composed of multiple components:
- Systematic Rewards: Objective checks, such as “Does the generated code run?” or “Is the syntax correct?”.
- Business KPIs: Direct business outcomes can be used as a reward. For example, a medical supply company maximized the “containment rate” of its customer support system—the percentage of calls handled end-to-end by the AI.
- LLM-as-a-Judge: For more nuanced, open-ended criteria like tone of voice or adherence to business guidelines, another LLM can act as a judge. The human’s task is simply to define the rubrics and system prompt for this judge. This is an activity that takes minutes or hours, not the weeks required for traditional annotation.
Overcoming the Complexity of RL
The primary challenge of RL is its inherent complexity. For instance, a popular algorithm like PPO requires orchestrating four large language models simultaneously. This is where an RL Ops platform like the Adaptive Engine becomes essential. It abstracts away the algorithmic and infrastructure complexity by providing pre-built recipes, allowing teams to focus on defining the business logic and rubrics for their specific use case while leveraging the best open-source models available.