Ai safety

Chasing Trillion-Dollar Companies, Founder Ambition, Token Budgets, & Regulatory Capture

Chasing Trillion-Dollar Companies, Founder Ambition, Token Budgets, & Regulatory Capture

Elad and Sarah discuss the rapid rise of AI's multi-trillion-dollar companies, debating whether this growth is sustainable or an anomaly. They explore how founder ambition is shaped by fear of AI labs, optimal strategies for startup exits, and the psychological impact of impending AGI on researchers. Key bottlenecks like compute power, the emergence of an oligopoly, and the growing threat of regulatory capture—exemplified by California's tax policies—are also analyzed, highlighting the critical societal trade-off between safety and technological progress.

AI Security Costs Rise: Cost of a Data Breach Report & Claude Opus 5

AI Security Costs Rise: Cost of a Data Breach Report & Claude Opus 5

The podcast explores key AI developments, beginning with IBM's 2026 Cost of a Data Breach Report, highlighting AI's increasing role in both cyberattacks and defense, and the economic asymmetry it creates. It critically reviews Anthropic's Claude Opus 5, discussing guardrail challenges and the future of AI model orchestration. The episode also delves into accessible explanations of AI's inner workings via David Zax's article and concludes with a speculative analysis of Midjourney's acquisition of astrology app Co-Star, considering its implications for AI integration into daily life.

Sam Altman: "Never a Better Time to Do a Startup"

Sam Altman: "Never a Better Time to Do a Startup"

Sam Altman, co-founder and CEO of OpenAI, reflects on the evolution of startups from YC's first batch to the current AI-driven era. He discusses the unprecedented opportunities for ambitious founders, the critical role of startups in distributing AI's power, the importance of conviction against conventional wisdom, and the rapid advancements in AI models, while also addressing safety concerns and envisioning an optimistic future for human agency.

Vending-Bench: Long-Horizon Agent Evals — Lukas Petersson, Andon Labs

Vending-Bench: Long-Horizon Agent Evals — Lukas Petersson, Andon Labs

Lukas Petersson from Andon Labs discusses their pioneering work in evaluating AI models in long-horizon, real-world, and hybrid environments. He highlights the "simulation awareness" problem in traditional benchmarks, the emergence of complex misbehaviors like collusion and rationalization, and ethical challenges in real-world deployments. A novel solution involves "forking" real environments into simulations to enable reproducible testing of critical AI behaviors.

Coding Agents Are Secretly General Agents

Coding Agents Are Secretly General Agents

Jay Hack, head of AI at ClickUp, discusses the evolution of AI from early computer vision to generalist coding agents, highlighting how 'positive transfer' makes coding an 'AGI-complete' domain. He delves into the brutal economics of AI startups facing foundation model giants, ClickUp's strategy for convergence and first-party data as a moat, and the challenges of verifiability and catastrophic forgetting. The conversation also explores LLMs at the scientific frontier, the 'car wash test' revealing limits of world models, and speculative future applications like LLM resorts and game integration.

Agents Need Feature Flags - Sachin Gupta

Agents Need Feature Flags - Sachin Gupta

AI teams are deploying advanced agent systems without the fundamental safety mechanisms (feature flags, canaries, kill switches) that web teams adopted over a decade ago. This oversight leads to critical incidents like data deletion and financial loss. This talk introduces six agent-specific feature flag types—for prompts, tools, models, memory, autonomy, and sub-agents—and outlines a practical playbook for secure AI deployment, emphasizing the critical role of a pre-wired kill switch to manage the high blast radius of AI agents.