Sudip Roy, Co-founder & CTO of Adaption Labs, discusses how "Adaptation" using gradient-free, inference-time techniques can solve the last 5% reliability gap that stalls enterprise AI adoption, offering a more dynamic and cost-effective alternative to traditional fine-tuning or simply waiting for the next frontier model.
Emmanuel Ameisen of Anthropic's interpretability team explains the inner workings of LLMs, drawing analogies to biology. He covers surprising findings on how models plan, represent concepts across languages, and the mechanistic causes of hallucinations, offering practical advice for developers on evaluation and post-training strategies.
Traditional benchmarks and leaderboards are insufficient for production AI. This summary details a practical, multi-layered evaluation strategy, moving from foundational system performance to factual accuracy and finally to safety and bias, using open-source tools like GuideLLM, lm-eval-harness, and Promptfoo.