Tokenless

Model reliability

Inside 847 Production Clinical AI Notes — Sebastian Fox, Composo

Inside 847 Production Clinical AI Notes — Sebastian Fox, Composo

Sebastian Fox, a medical doctor and AI evaluation expert, dissects the critical problem of subtle yet dangerous errors in AI-generated clinical notes within high-stakes healthcare. He reveals why conventional AI verification methods fail to grasp the nuanced concept of "what matters" and introduces a novel, adaptive evaluation framework that continuously learns from real outputs and expert judgment to build a dynamic, case-specific standard for AI reliability.

AI That Learns While You Use It

AI That Learns While You Use It

Sudip Roy, Co-founder & CTO of Adaption Labs, discusses how "Adaptation" using gradient-free, inference-time techniques can solve the last 5% reliability gap that stalls enterprise AI adoption, offering a more dynamic and cost-effective alternative to traditional fine-tuning or simply waiting for the next frontier model.

Inside the AI Black Box

Inside the AI Black Box

Emmanuel Ameisen of Anthropic's interpretability team explains the inner workings of LLMs, drawing analogies to biology. He covers surprising findings on how models plan, represent concepts across languages, and the mechanistic causes of hallucinations, offering practical advice for developers on evaluation and post-training strategies.

Strategies for LLM Evals (GuideLLM, lm-eval-harness, OpenAI Evals Workshop) — Taylor Jordan Smith

Strategies for LLM Evals (GuideLLM, lm-eval-harness, OpenAI Evals Workshop) — Taylor Jordan Smith

Traditional benchmarks and leaderboards are insufficient for production AI. This summary details a practical, multi-layered evaluation strategy, moving from foundational system performance to factual accuracy and finally to safety and bias, using open-source tools like GuideLLM, lm-eval-harness, and Promptfoo.