Tokenless

Model orchestration

Preferences Over Benchmarks: Model Routing — Archana Kamath & Tyler Gillam, DigitalOcean

Preferences Over Benchmarks: Model Routing — Archana Kamath & Tyler Gillam, DigitalOcean

This talk argues against the common practice of picking LLMs based solely on leaderboards, emphasizing that there's no single best model, only the right one for a given request. It introduces Digital Ocean's Inference Router, a customizable, open-source solution that intelligently selects models based on user-defined preferences (cost, latency, task, quality) rather than benchmarks, demonstrating significant cost savings and performance improvements in live demos.

Stripe buys OpenRouter, Ramp’s AI Index & IBM’s OpenAI deal

Stripe buys OpenRouter, Ramp’s AI Index & IBM’s OpenAI deal

This episode explores the dynamic landscape of the AI industry, dissecting IBM's strategic alliance with OpenAI for enterprise AI integration, Stripe's acquisition of OpenRouter to capitalize on AI token routing, and insights from Ramp's AI Index revealing evolving AI spend and the rise of open-source models. It highlights the shift from model-centric to infrastructure and governance-centric AI, and touches on the ethical implications of AI-drafted legislation.

Scaling the Next Paradigm of Heterogeneous Intelligence — Adrian Bertagnoli, Callosum

Scaling the Next Paradigm of Heterogeneous Intelligence — Adrian Bertagnoli, Callosum

Adrian Bertagnoli from Callosum argues that the era of scaling monolithic models on homogeneous GPU clusters is ending. He introduces "heterogeneous intelligence," a new paradigm where model architectures, chip types, and workflows are optimized together. By routing subtasks to the most efficient model and hardware, this approach achieves significant performance gains, as demonstrated by two key results: a 7x cost reduction in recursive reasoning tasks using Cerebras, and state-of-the-art performance on the Video Web Arena benchmark, outperforming leading GPT and Gemini models at a fraction of the cost and time.

AI Code Generation: Wins, Fails and the Future

AI Code Generation: Wins, Fails and the Future

A panel of experts discusses the future of AI in software engineering, focusing on the "barbell effect" where AI excels at hyper-complex tasks but fails at simple ones. The conversation explores whether performance is driven by the model or the agent orchestration, the evolving role of the engineer as an architect, and the significant challenges open-source tools face against vertically integrated proprietary systems, particularly the high cost of inference.