Chip Huyen, an AI expert and author of 'AI Engineering', explains the realities of building successful AI applications. She covers the nuances of model training, the critical role of data quality in RAG systems, the mechanics of RLHF, and why the future of AI improvement lies in post-training, system-level thinking, and solving UX problems rather than just chasing the newest models.
Building successful AI applications requires a sophisticated engineering approach that goes beyond prompt engineering. This involves creating intentionally engineered evaluations (evals) that reflect user feedback, focusing on "context engineering" to optimize tool definitions and outputs, and maintaining a flexible, model-agnostic architecture to adapt to the rapidly evolving AI landscape.
The field of AI Engineering is evolving from simple 1:1 applications to complex, AI-intensive systems with high LLM-call ratios. This talk explores the search for a 'Standard Model' for AI engineering, analogous to MVC or ETL in traditional software, proposing several candidates including LLM OS, LLM SDLC, and a new SPADE (Sync, Plan, Analyze, Deliver, Evaluate) model for building robust applications.
Omar Khattab, creator of DSPy, reinterprets the 'Bitter Lesson' for AI engineering, arguing that the key to building robust and enduring AI systems is to move beyond brittle prompt engineering. He advocates for a declarative, modular approach that separates the fundamental program logic from the rapidly changing landscape of LLMs, optimizers, and inference techniques.
Moving an AI PoC from 50% to 100% reliability requires a new development paradigm. This talk introduces a practical, evaluations-first approach, reverse-engineering tests from real-world user scenarios and business outcomes to build a robust benchmark, prevent regressions, and enable confident optimization.
Barr Yaon of Amplify Partners presents early findings from the 2025 State of AI Engineering survey, covering LLM usage, customization techniques like RAG and fine-tuning, the state of AI agents, key challenges like evaluation, and community perspectives on the future of AI.