A Playground for AI Engineers

Hotmart, a leading company in the creator economy, has transitioned from a traditional machine learning company to one that builds AI-powered products, including “Agent as a Product”. This shift involves leveraging both classic machine learning models and modern Large Language Models (LLMs) to solve distinct problems, requiring a significant evolution of their MLOps platform.

Blending Classic ML with Modern LLMs

While LLMs are powerful, Hotmart continues to use classic NLP models like those built with spaCy and scikit-learn for specific, high-volume backend tasks. A primary example is the classification and entity extraction from thousands of daily customer support tickets.

The decision to stick with these models is based on several key factors:

• Cost and Speed: For tasks like ticket classification, using an LLM is “overkill”. A classic model running as a microservice on Kubernetes is significantly faster and cheaper, avoiding high token costs.
• Performance on Niche Languages: Hotmart’s user base is predominantly Portuguese-speaking. They found that their custom-trained spaCy models outperformed LLMs and even small language models (SLMs), which tend to struggle with multilingual capabilities compared to their English performance.
• Tradeoffs: While classic models require a continuous training pipeline to manage concept and data drift, the overall cost-benefit analysis still favors them for these specific, high-throughput use cases.

For user-facing applications, however, LLMs are preferred for their ability to provide a more human-like interaction. This creates a hybrid system where different tools are used for different parts of the same business problem (e.g., classic ML for backend analytics on support tickets, and an LLM-powered agent for direct customer interaction).

AI Agents as a Core Product Offering

Hotmart has successfully launched several AI agents that create direct value for creators and their students.

• Hotmart Tutor: This is a 24/7 AI-powered teacher available within a creator’s online course.

  • It operates using a Retrieval-Augmented Generation (RAG) architecture, ensuring that it only answers questions based on the creator’s own content (videos, ebooks, etc.). This prevents hallucinations and keeps the information contextually relevant.
  • For students, it transforms the learning experience from a passive one (watching a video) to an active one (asking questions and interacting), which has led to significantly higher retention and engagement rates compared to courses without the tutor.
  • For creators, it provides invaluable insights into what their students are struggling with, revealing opportunities to create new content modules or clarify existing ones.

• Agent as a Product: Building on the success of the tutor, Hotmart now allows creators to build and sell their own AI agents. Creators can package their unique knowledge base into an interactive agent, creating a new product and upsell opportunity beyond a standard online course.

• AI Sales Agent: To tackle the problem of abandoned shopping carts, Hotmart developed an AI sales agent that interacts with potential customers via WhatsApp.

  • The agent proactively reaches out to users who didn’t complete a purchase, helps overcome their objections (e.g., questions about payment security, course content), and guides them to complete the sale.
  • It can facilitate the purchase directly within WhatsApp, creating a seamless experience.
  • This initiative has led to a significant improvement in revenue for creators by recovering sales that would have otherwise been lost.

Evolving the MLOps Platform for the Generative AI Era

The rise of generative AI required a fundamental shift in Hotmart’s MLOps platform, which was originally built for classic ML lifecycles.

When you work with AI agent platforms, you need to be ready to change everything that you created so far. The field is moving so fast.

The platform had to evolve from focusing on feature stores and experiment tracking to incorporating new, essential components for building agents:

• Vector Databases: A critical component for RAG that was not a priority for most teams before 2023.
• Guardrails and Fallbacks: Mechanisms to ensure agent safety and reliability, such as having a backup LLM provider in case the primary one fails.
• Modularity: The key architectural lesson was to build a highly modular platform where each component is independent. This allowed them to, for example, switch from a third-party vector database vendor to an in-house solution by only changing an endpoint, without disrupting the entire system. This flexibility is crucial to adapt to rapid industry changes, such as the emergence of new agent protocols.

Agent Tooling and Context Engineering

The agents are empowered through a combination of tooling and what is described as context engineering. To function effectively, an agent needs the right information at the right time. This is achieved by:

• Using a low-latency database to pre-populate static context, such as a user’s name and the course they are interested in.
• Dynamically building context on the fly by querying knowledge bases (like vector stores) to answer specific questions.

This careful construction of context for each interaction is fundamental to making the agent’s responses accurate, relevant, and fluid.

• Tags:
• AI Agents
• RAG
• Feature
• MLOps
• LLMOps
• Product Strategy