An expert overview of post-training techniques for language models, covering the entire workflow from data generation and curation to advanced algorithms like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Reinforcement Learning (RL), along with practical advice on evaluation and iteration.
This talk explores the limitations of current methods for providing knowledge to LLMs, such as large context windows and Retrieval-Augmented Generation (RAG). The speaker argues that the future lies in training knowledge directly into the model's weights. This is achieved through a combination of generating large synthetic datasets from small amounts of source material and using parameter-efficient fine-tuning (PEFT) techniques like LoRA to avoid catastrophic forgetting. The goal is to create more capable, personalized, and efficient models by fundamentally altering how they store and access information.
Bhuvaneswari Subramani details the "Agentic Shift" in AI by presenting an evolutionary journey through seven foundational system design patterns. The talk progresses from simple conversational clients to sophisticated, multi-agent systems, covering key patterns like Retrieval-Augmented Generation (RAG), Self-Correcting RAG, and the Model Context Protocol (MCP), explaining how each pattern adds new layers of context, action, and autonomy.
A comprehensive overview of the entire process behind building Olmo 3 Think, covering the full stack from pre-training architecture and data selection to the detailed post-training recipe involving SFT, DPO, and a deep dive into the advanced infrastructure for scaling Reinforcement Learning (RL). The summary also includes critical reflections on the challenges and nuances of evaluating modern reasoning models.
A deep dive into the challenges and solutions for efficient Reinforcement Learning (RL) in enterprise settings. The talk contrasts synchronous and asynchronous RL, explains the critical trade-off of "staleness" versus stability, and details a first-principles system model used to optimize GPU allocation for maximum throughput.
Yusuf Olokoba, founder of Muna, details a compiler-based approach to transform Python AI functions into self-contained native binaries. This talk explores the technical pipeline, including custom AST-based tracing, type propagation, and the strategic use of LLMs for code generation, enabling a universal, OpenAI-style client for running any model on any platform.