A walkthrough of the memory wall, explaining why raw chip compute growth has not translated into proportional AI workload speedups. It covers arithmetic intensity, the roofline model and ridge points for the B200 GPU, why LLM decode is memory-bound while prefill is compute-bound, and how batching and continuous batching shift workloads toward higher efficiency.
A hands-on workshop from Langfuse on evaluating AI applications where mistakes are costly. Using a banking assistant as the running example, Annabell Schäfer and Doneyli De Jesus walk through the AI engineering loop: tracing with PII masking, building datasets, turning domain-expert comments into failure modes, mixing LLM-as-a-judge and code evaluators, and CI gates that block prompt changes when scores drop.
Nick Nisi and Zack Proser from WorkOS explore how to move "Beyond Done" with AI agents, ensuring their work is truly complete and reliable. They demonstrate building robust verification loops using custom CLIs, browser tools, and LLM judges, allowing agents to self-check and self-correct, preventing costly mistakes and accelerating development velocity for a technical audience.
A panel of experts discusses the challenges of granting autonomy to AI agents due to coarse-grained authorization systems. They advocate for intent-based runtime authorization, leveraging concepts like "missions" and "budgets" from AOTH, brain-hands architecture, and dynamic policy management. The discussion covers agentic commerce, accountability through tamper-resistant audit logs, and whether P(doom) is a real threat, emphasizing engineering ingenuity and societal learning from failures.
This episode of Paper Club addresses AI safety from a scientific perspective, contrasting it with recent media sensationalism. Stanford professors Clark Barrett and Mykel Kochenderfer discuss how automated reasoning can mathematically prove code security, how aircraft stress-testing methods are being adapted for language models, and the potential need for an NTSB-like agency for AI, highlighting the real-world engineering challenges and solutions for AI safety.
Eugene Cheah, CEO of Featherless AI, presents a four-step framework for cutting AI inference costs at scale: measure token spend by team and workload, right-size models through evals, classifiers, and fine-tuning, buy GPU capacity with an understanding of the speed-versus-cost tradeoff, and check whether organizational processes can absorb the extra output.