Inside the Race to Measure Frontier Intelligence

Independent model evaluation is crucial for AI progress, as public benchmarks are saturated and models optimize for them. Rayan Krishnan discusses VALS's role in providing continuously evolving, third-party evaluations, highlighting the Llama 4 'disaster' and the need to measure increasingly agentic systems. The conversation covers the complexities of evaluation beyond capability to include cost and latency, the looming financial impact of token spend on enterprises, and the critical role of evaluations in informing AI policy and international coordination for aligning AI with human intent and managing geopolitical risks, especially concerning recursive self-improvement.