Jun 30, 2026

Why Hardware-Software Co-Design Is AI's Real 100x: Dylan Patel of SemiAnalysis

Dylan Patel, founder of SemiAnalysis, details how software-hardware co-design drives 100x gains in AI performance, citing examples like DeepSeek's optimization for Nvidia Hopper and the architectural differences between OpenAI's sparse and Anthropic's dense models. He introduces InferenceX, a living benchmark tracking a 60x annual cost reduction in AI quality, and explains why the "CUDA moat" narrative is shifting. Patel also delves into the persistent compute crunch, the strategic rise of neoclouds, and Jensen Huang's efforts to foster a multipolar AI world to prevent hyperscaler dominance.

Jun 02, 2026

Task Fidelity Scaling Laws — Kobie Crawdord, Snorkel

An experiment by Snorkel AI reveals that in agentic AI training, the quality of tasks is paramount. Using the same model and compute, fine-tuning on high-quality tasks yielded a 6% performance improvement, a 5x greater uplift compared to the 1% gain from low-quality tasks. The key difference lies in the nature of the tasks: high-quality tasks are genuinely harder, featuring more tool calls and cleaner failure modes that provide a meaningful learning signal. In contrast, low-quality tasks often fail due to ambiguity and environmental noise, hindering effective model improvement.

May 25, 2026

Agentic Evaluations at Scale, For Everybody — Nicholas Kang & Michael Aaron, Google DeepMind

Nicholas Kang and Michael Aaron from Google DeepMind's Kaggle team discuss the broken state of AI evaluations—scattered, non-transparent, and created by a homogenous group. They present their solutions: a community-driven benchmarks platform, a PvP Game Arena for non-saturating ELO ratings, standardized agent exams, and hackathons to crowdsource novel evals and address the limitations of current benchmarking practices.

Mar 09, 2026

OpenClaw's Memory Sucks and the fix is simple — Dhravya Shah, Supermemory

Dhravya Shah, founder of Super Memory, details the evolution of his company from a simple RAG-based consumer app to a sophisticated, open-source context infrastructure for AI, and introduces a novel hooks-based memory solution for OpenClaw.

Dec 20, 2025

Are AI Benchmarks Telling The Full Story? [SPONSORED]

AI models are often benchmarked like Formula 1 cars, excelling on technical exams but failing the test of daily human experience. Researchers Andrew Gordon and Nora Petrova from Prolific critique the 'leaderboard illusion' of current ranking systems and introduce their HUMAINE leaderboard, a new framework that uses census-based sampling and the TrueSkill algorithm to measure how helpful, safe, and relatable models are to real people, not just tech enthusiasts.

Oct 18, 2025

Why AI Needs Culture (Not Just Data) - Prolific [Sponsored]

Sara Saab and Enzo Blindow from Prolific discuss the critical, and growing, need for high-quality human evaluation in the age of non-deterministic AI. They explore the limitations of current benchmarks, the dangers of agentic misalignment as revealed by Anthropic's research, and how Prolific is building a "science of evals" by treating human feedback as a robust infrastructure layer.