The Truth About LLM Training

Paul van der Boor and Zulkuf Genc from Prosus discuss the practical realities of deploying AI agents in production. They cover their in-house evaluation framework, strategies for navigating the GPU market, the importance of fine-tuning over building from scratch, and how they use AI to analyze usage patterns in a privacy-preserving manner.

August 9, 2025 6 min read

The Truth About LLM Training — source: https://www.youtube.com/watch?v=QfwJYpDlslw

At Prosus, the goal is to bring AI agents into production in a way that is scalable, reliable, and tailored to specific, real-world use cases. This involves a multi-faceted approach that spans from rigorous model evaluation to sophisticated infrastructure management and the development of a proprietary “AI workforce.”

The Cornerstone: A Robust, In-House Evaluation Pipeline

Standard industry benchmarks are often poor indicators of a model’s performance on specialized tasks. To address this, Prosus developed an internal evaluation platform, proslm.ai, to systematically test models against their own use cases.

Custom Eval Sets: The evaluation sets are crucial and are not public. They are designed to test models on tasks relevant to Prosus’s portfolio companies, such as summarizing calls for Polish car dealers or predicting ingredients for Brazilian recipes. The data needs to be constantly refreshed, as eval sets can “get rotten very quickly,” with a suggested refresh cycle of every 3-6 months.
Rapid Benchmarking: When a new model like DeepSeek is released, it can be run through the evaluation pipeline, and within hours, the team knows exactly how it performs on key tasks like function calling or summarization, what it costs, and where it ranks against hundreds of other models. This provides a clear signal amidst the market hype.
Eval-Driven Development: New AI projects start from the end: creating the evaluation for the task first. This establishes a clear baseline and target. The proslm.ai platform now includes private spaces for portfolio companies to benchmark models against their own proprietary data.

Creating High-Quality Eval Sets: Pizza and Labeling Parties

The biggest challenge in evaluation isn’t the tooling, but creating the evaluation sets themselves. Prosus has found a practical and engaging solution:

“We’ll gather a whole group of people that are hungry and we organize a labeling party. We provide them some pizza and some other snacks and we put them in a room and explain… And we sit down and we manually with, you know, fueled by pizza, curate these eval sets.”

This manual curation is laborious but essential for creating high-quality, domain-specific data. It also serves as a great way to give non-AI team members exposure to the work being done.

LLM as a Judge: Automating and Scaling Evaluation

To speed up the evaluation process, the team uses an “LLM as a judge.”

Methodology: The system provides a model with the ground truth (or a highly-rated answer, as in Stack Overflow’s case), the question, and the new model’s generated answer. The judge model then assesses the correctness. This has achieved around 90% accuracy.
Rubric Design: Instead of a simple 0-10 score, they developed a more practical, four-point scale (0-3) tailored to the user’s goal. For a technical question, what matters is if the answer is good enough to stop the user from searching further. A score of ‘2’ or ‘3’ means the answer is acceptable or perfect, solving the user’s problem.
Benchmarking the Judges: Not all models are good judges. The team runs benchmarks to determine the best model for the judge role itself. Currently, GPT-4.o is their preferred judge, but it can be swapped out as better models emerge.

The Shifting Model Landscape and GPU Headaches

The LLM space is evolving at an incredible pace, with the best model for a given task changing constantly. Paul van der Boor describes models as “the quickest depreciating assets, right? They’ve got a half life and the most expensive depreciating assets.”

This rapid depreciation has two major implications:

Model Agnosticism: The team must be agnostic about where a model comes from. While OpenAI was once the clear leader, the margin has narrowed significantly. Open-source models from Mistral, DeepSeek, and various Chinese developers are now highly competitive and are being used in production.
Infrastructure Strategy: Deciding to use an open-source model necessitates a strategy for inference. The choice is between using a token-based API service from a provider like Together AI (for vanilla open-source models with privacy guarantees) or self-hosting.

When self-hosting for training or inference, procuring GPUs presents a major challenge. The team needs spiky, on-demand capacity for training runs but often faces providers pushing for long-term, expensive commitments for reserved capacity. Key considerations when choosing a GPU provider include:

Flexibility: The ability to get on-demand capacity without prohibitive costs.
Software Stack: The provider must support the required frameworks (e.g., Hugging Face integrations) and not lock them into a proprietary middleware that lags in supporting new models.
Support: High-quality, responsive support is critical, as running large GPU clusters is complex and things frequently go wrong.
Data Handling: Clear guarantees on data privacy, security, and management are non-negotiable.

Fine-Tuning as a Core Strategy

To leverage valuable, proprietary domain data, fine-tuning is essential. The team is not focused on training base models from scratch. Instead, they build data and training pipelines that can be applied on top of the best available base models.

“What depreciates is the base model. So we are not interested in creating a language model from the scratch… if the base model depreciates… we can just replace the base model and then we continue and just enjoy the performance gains.”

This approach allows them to create highly specialized models (e.g., a large commerce model) that outperform generic ones on specific tasks, while also being more cost-effective at scale. Within the next 12-18 months, they expect to displace a significant amount of commercial model traffic with their own proprietary, fine-tuned models.

Understanding Usage with a Privacy-First AI Workforce

To understand how internal tools like their “Token” productivity assistant are used without violating strict privacy constraints (SOC 2 compliance), the team built an AI data analyst agent named “Float.”

Automated Tagging: Float analyzes entire conversations (user questions, model answers, follow-ups) and applies around 25 different tags to classify the domain (e.g., Finance, HR), task type (e.g., coding, debugging), and even estimates the time saved by the user.
LLM-Powered Clustering: To manage the “wild” and inconsistent nature of open-ended tags generated by the LLM, they developed an LLM-powered clustering system. Instead of feeding all tags to a model at once (which fails after ~100 tags), they create an initial hierarchy of clusters and then use an LLM to refine and name them, creating a multi-level hierarchy for both high-level and granular analysis.

This pattern of using AI agents to analyze large volumes of data is being expanded into an “AI workforce,” with data analyst agents being connected to various internal datasets to provide insights for sales, customer support, and more.