Model performance

State of Data — Sean Cai, Independent / State of Data

State of Data — Sean Cai, Independent / State of Data

Sean Cai discusses the evolving landscape of AI data markets, highlighting the shift from raw annotation to high-quality, process-based data. He introduces "Verifier's Law" and its three axes to explain application layer maturity, critiques the shortcomings of current benchmarks, and predicts future AI trends by analyzing data market signals. Cai concludes by envisioning data companies transforming into "neo-labs" that provide enterprise-level reinforcement learning as a service and "Antikythera mechanisms" to manage and monetize real-world data assets.

What Do Models Still Suck At? - Peter Gostev, Arena.ai, BullshitBench

What Do Models Still Suck At? - Peter Gostev, Arena.ai, BullshitBench

Despite benchmarks showing relentless progress, many users remain dissatisfied with LLM responses in real-world scenarios. This summary explores two key analyses—a custom 'nonsense question' benchmark and trends from Chatbot Arena's 'dislike both' data—to reveal the persistent gaps in model reasoning, reliability, and domain-specific understanding.