Scaling the Next Paradigm of Heterogeneous Intelligence — Adrian Bertagnoli, Callosum
Adrian Bertagnoli from Callosum argues that the era of scaling monolithic models on homogeneous GPU clusters is ending. He introduces "heterogeneous intelligence," a new paradigm where model architectures, chip types, and workflows are optimized together. By routing subtasks to the most efficient model and hardware, this approach achieves significant performance gains, as demonstrated by two key results: a 7x cost reduction in recursive reasoning tasks using Cerebras, and state-of-the-art performance on the Video Web Arena benchmark, outperforming leading GPT and Gemini models at a fraction of the cost and time.
Adrian Bertagnoli, a founding engineer at Callosum, outlines a fundamental shift in scaling artificial intelligence, moving from the current paradigm of “homogeneous intelligence” to a more efficient and powerful “heterogeneous intelligence.”
The Prevailing Paradigm: Homogeneous Intelligence
The current approach to scaling AI is defined by homogeneity:
- Architecture: Training single, large, dense models.
- Workflow: Relying on single Large Language Model (LLM) calls.
- Hardware: Running these models on large, uniform clusters of identical chips (e.g., GPUs).
This paradigm was born from the discovery of neural scaling laws, which showed that performance improves with more data and parameters. However, this is primarily a training-centric view and is becoming less relevant and efficient in an inference-dominated world.
The Next Paradigm: Heterogeneous Intelligence
The future of AI scaling lies in heterogeneity, where diverse components co-evolve and are orchestrated to work together optimally. This shift is already underway:
- Architectures: Mixture-of-Experts (MoE) models are replacing dense models.
- Workflows: Multi-agent systems are replacing single LLM calls.
- Hardware: Disaggregated systems with different chips for different purposes (e.g., pre-fill vs. decode) are emerging.
The core principle is that real-world problems are complex and decompose into sub-problems that require different types of intelligence. Scaling a single, monolithic “generalist” model to solve every sub-task is highly inefficient. Instead, heterogeneous systems orchestrate models of different architectures and sizes, running on optimal hardware, to tackle problems more effectively. Callosum aims to build the automation layer that routes tasks to the right chip and model without bespoke, manual decisions.
Case Study 1: Heterogeneous Recursion
A key challenge in long-context reasoning is “context rot,” where model performance degrades significantly as information complexity increases, even when not using the full context window. The concept of recursive language models addresses this by treating the context as an environment and having an agent programmatically interact with it to extract smaller, relevant sub-contexts.
Callosum extended this concept by introducing heterogeneity. Instead of passing the sub-context to an identical agent, their system routes it to different, specialized models running on optimal hardware. The results, benchmarked on Ulong against a frontier model, are dramatic:
- On Cerebras: 7 times cheaper and 5 times faster.
- On SambaNova: 12 times cheaper and 3 times faster.
This demonstrates that by making intelligent architectural decisions about where to route a workload, it’s possible to match the intelligence of a frontier model while drastically reducing cost and latency.
Case Study 2: Visual Web Navigation
On the state-of-the-art Video Web Arena benchmark, Callosum’s heterogeneous approach outperformed the leading GPT and Gemini models by 18% and 25%, respectively. The key was acknowledging that web navigation is not a monolithic task; it decomposes into sub-tasks like visual reasoning, textual analysis, and simple actions like zooming.
By creating a mixture of models (e.g., Qwen 3 VL8B and Kimi K2.5), they achieved superior performance and efficiency:
- A mixture of Qwen and GPT was 3 times faster and 3.7 times cheaper than using the GPT model alone.
- The system offloaded simple sub-tasks like zooming and visual parsing to smaller, less powerful models. This single optimization made those specific steps 11 times faster and 43 times cheaper.
These granular savings accumulate, leading to the significant overall performance improvement and showcasing a fundamental shift of the Pareto frontier.
The Future of Compute
The evolution of compute can be seen in three eras:
- CPU Era: Making compute faster.
- GPU Era: Making compute massively parallel.
- Heterogeneous Era: Making compute diverse and adaptable, optimally mapping multi-agent workloads onto a variety of specialized chips.
Callosum is actively building this future, working with the UK’s ARIA institute on the first heterogeneous collocated cluster. The era of homogeneous scaling delivered incredible progress, but the next wave of innovation will come from heterogeneous intelligence, where models, workflows, and silicon co-evolve to create systems that are collectively smarter, faster, and cheaper.