Posts

MCP Apps: Extending the Frontier — Ido Salomon & Liad Yosef

MCP Apps: Extending the Frontier — Ido Salomon & Liad Yosef

MCP Apps transforms chat and coding assistant interactions from text-heavy responses into rich, interactive user interfaces, preserving brand identity and enhancing user experience. It allows services to send "UI atoms" directly into assistants like Claude and ChatGPT, enabling a "write once, run anywhere" model across a massive user base. This standard defines how hosts render web components, manage user interactions, and shifts control of the user journey to the agent, ushering in a new era of the agentic web.

MCP Tasks (async): Why Aren't Any Agents Supporting Them? — Cornelia Davis, Temporal

MCP Tasks (async): Why Aren't Any Agents Supporting Them? — Cornelia Davis, Temporal

Cornelia Davis explains why the MCP tasks specification, designed for durable, long-running, and interactive operations, hasn't seen widespread adoption. She details V1's architectural flaws, particularly concerning scalability and stateful human-in-the-loop interactions, and introduces V2's shift to a stateless core, client-initiated updates, and a clearer path for client-side durability, while outlining future work on scaling and adoption.

When Will The Benchmaxxing Plague End? — Nick Heiner, Surge AI

When Will The Benchmaxxing Plague End? — Nick Heiner, Surge AI

Nick Heiner explores the phenomenon of "benchmaxing" in AI, where models are optimized for benchmark scores rather than real-world utility. He exposes common antipatterns in benchmark creation, such as contamination, reward hacking, and misaligned verifiers, and critiques labs' tactics like gaming leaderboards. Heiner advocates for a higher standard, emphasizing the need for human expertise, high-fidelity data, and rigorous alignment in evaluation to ensure benchmarks genuinely reflect AI's value.

This CPO regrets that product management exists | Tom Verrilli (CPO of Whatnot)

This CPO regrets that product management exists | Tom Verrilli (CPO of Whatnot)

Tom Verrilli, CPO at Whatnot, discusses his controversial stance that "we regret product management exists," advocating for a leaner, more hands-on PM approach. He delves into how Whatnot structures its product team, its rigorous hiring process focused on deep systems thinking and IC work, and the transformative impact of AI on product and data science functions. Verrilli also shares mental models for balancing strategy and iteration, navigating the CPO-founder relationship, and key lessons from his time at Twitter, emphasizing the importance of core PM skills in an AI-driven world.

Understanding AI Agent Hallucination in AI Systems

Understanding AI Agent Hallucination in AI Systems

Learn about AI hallucinations, why they occur in autonomous agents, and how they pose new risks as AI takes action. Discover key mitigation strategies including data grounding, tool-based reasoning, scope control, and human-in-the-loop interventions to ensure reliable AI performance.

Teaching AI to Find Real Vulnerabilities — David Brumley, Bugcrowd

Teaching AI to Find Real Vulnerabilities — David Brumley, Bugcrowd

David Brumley discusses the challenges and solutions for teaching AI models to hack, drawing parallels with human learning. He introduces a 'ladder of tasks' approach for reinforcement learning and addresses the critical flaw of traditional benchmarks: measurement difficulties with multiple vulnerabilities and 'reward hacking.' His team's 'Audit Task' uses deterministic graders and precision/recall metrics for open-world assessment. He demonstrates this with an in-depth case study on attacking Chrome's V8 engine, showcasing how advanced models achieve real zero-day exploits, and warns against 'benchmaxxing security' without robust, honest grading.