An expert overview of post-training techniques for language models, covering the entire workflow from data generation and curation to advanced algorithms like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Reinforcement Learning (RL), along with practical advice on evaluation and iteration.
This talk introduces Anaximander, a system designed to bridge the gap between traditional, GUI-driven Geographic Information System (GIS) workflows and modern, code-heavy machine learning practices. Anaximander integrates geospatial foundation models directly into QGIS, allowing experts to interactively orchestrate, run, and evaluate models for tasks like semantic segmentation and object detection on satellite imagery.
At Applied Compute, efficient Reinforcement Learning is critical for delivering business value. This talk explores the transition from inefficient synchronous RL to a high-throughput asynchronous 'Pipeline RL' system. The core challenge is managing 'staleness'—a side effect of in-flight weight updates that can destabilize training. The speakers detail their first-principles systems model, based on the Roofline model, used to simulate and find the optimal allocation of GPU resources between sampling and training, balancing throughput with algorithmic stability and achieving significant speedups.
a16z co-founders Marc Andreessen and Ben Horowitz discuss the shift to a decentralized media ecosystem, their investment thesis on supply-driven markets, and the transformative impact of AI. They detail the a16z model of leveraging reputation as a core asset to turn inventors into CEOs and explain why AI represents a fundamental reinvention of computing that will unlock unprecedented growth.
Ben Luria, CEO of Hirundo, discusses the critical need for machine unlearning, framing it as a form of "AI neuro-surgery" for enterprise AI. He explains how this technique directly modifies model weights to remove unwanted data and behaviors, addressing core risks that superficial solutions like guardrails cannot solve.
State Space Models (SSMs) are emerging as a powerful and efficient alternative to Transformers for handling sequential data. Aaron Baughman explains the core concepts of SSMs, their mathematical foundations, and how architectures like S4 and Mamba address the memory and scalability challenges inherent in Transformers, leading to a new generation of faster, more intelligent hybrid AI models.
Nikesh Arora, CEO of Palo Alto Networks, shares his unconventional journey and leadership philosophy. He provides a masterclass in building a multi-platform company through strategic M&A, explains why founders should sometimes ignore customers, and reveals how to lead with conviction while managing imposter syndrome.
Stewart Butterfield, co-founder of Slack and Flickr, shares the product frameworks and leadership principles that guided his success. He delves into concepts like "utility curves" for feature investment, the "owner's delusion" in product design, and why focusing on "comprehension" is often more important than reducing friction. He also introduces powerful mental models for organizational effectiveness, such as combating "hyper-realistic work-like activities" and applying Parkinson's Law to team growth.
Intuit CEO Sasan Goodarzi discusses the operational playbook for reinventing a 40-year-old company, from its slow transition to SaaS to its early adoption of AI. He shares insights on winning the SMB market by treating small businesses like consumers, building effective channel partnerships, and developing a platform strategy. Goodarzi also details his leadership philosophy, emphasizing that grit and curiosity are more critical than raw talent.
Experts discuss an OpenAI paper that reframes hallucinations as a feature driven by training incentives, not just a bug. The panel also revisits Dario Amodei's prediction on AI coding, explores AI's chaotic impact on the job market, and imagines the future of running LLMs on business-card-sized devices.
A conversation with C++ expert Patrice Roy, author of 'Memory Management in C++', exploring the core principles of modern memory management. The discussion covers the pitfalls of over-allocation, the nuanced use of smart pointers, the importance of measurement before optimization, and when to consider custom allocators for peak performance.
Jay Alammar, Director at Cohere, discusses the practical adoption of Large Language Models in the enterprise. He covers the evolution of Retrieval-Augmented Generation (RAG) from a simple anti-hallucination tool to complex, agentic systems, the critical role of evaluation as intellectual property, and future trends like text diffusion and the increasing capability of smaller models for specialized business tasks.
Co-founder David shares Recall.ai's journey from a YC hackathon project to the essential data infrastructure for over 1,000 AI companies. He covers their strategic pivot to an API, the fundraising grind, and the lessons learned in building a lean, high-agency team that powers the future of AI with conversation data.
a16z partners Chris Dixon and Anish Acharya discuss the exponential forces shaping technology, including network effects, composability, and Moore's Law. They explore how these principles apply to the age of AI, covering strategies for founders like 'come for the tool, stay for the network,' the nature of defensibility, investing in movements, the shift to native AI platforms, and the crucial role of open-source AI.
Birgitta Böckeler from Thoughtworks provides a comprehensive overview of the current AI tooling landscape for software delivery. The talk focuses on the architecture and capabilities of modern coding assistants, differentiating between the underlying model and the crucial role of tooling in providing context. It explores what works today, the promising but challenging frontiers like testing and refactoring, and offers a practical guide to the major tools in the ecosystem.
Get updates straight to your inbox. No spam, just useful content.