Tokenless

Ml

A 4-step framework for building delightful products | Nesrine Changuel (Spotify, Google, Skype)

A 4-step framework for building delightful products | Nesrine Changuel (Spotify, Google, Skype)

Nesrine Changuel, a former product leader at Spotify and Google, presents a pragmatic framework for building delightful products. She argues that delight is not a superficial add-on but a core business strategy that drives retention and differentiation by creating products that serve both functional and emotional needs. The summary covers her three pillars of delight, a four-step model for implementation, the 50-40-10 rule for prioritization, and real-world examples from her time at Chrome, Google Meet, and Spotify.

Trust at Scale: Security and Governance for Open Source Models // Hudson Buzby // MLOps Podcast #338

Trust at Scale: Security and Governance for Open Source Models // Hudson Buzby // MLOps Podcast #338

Hudson Buzby from JFrog discusses the critical security, governance, and legal challenges enterprises face when adopting open-source AI models. He highlights the risks lurking in repositories like Hugging Face and argues for a centralized, curated AI gateway as the essential framework for enabling safe, scalable, and cost-effective AI development.

Build Hour: Reinforcement Fine-Tuning

Build Hour: Reinforcement Fine-Tuning

A deep dive into Reinforcement Fine-Tuning (RFT), covering how to set up tasks, design effective graders, and run efficient training loops to improve model reasoning, based on a live demonstration from OpenAI's Build Hours.

Iterating on Your AI Evals // Mariana Prazeres // Agents in Production 2025

Iterating on Your AI Evals // Mariana Prazeres // Agents in Production 2025

Moving an AI agent from a promising demo to a reliable product is challenging. This talk presents a startup-friendly, iterative process for building robust evaluation frameworks, emphasizing that you must iterate on the evaluations themselves—the metrics and the data—not just the prompts and models. It outlines a practical "crawl, walk, run" approach, starting with simple heuristics and scaling to an advanced system with automated checks and human-in-the-loop validation.

Evaluation-Driven Development with MLflow 3.0

Evaluation-Driven Development with MLflow 3.0

Yuki Watanabe from Databricks introduces Evaluation-Driven Development (EDD) as a critical methodology for building production-ready AI agents. This talk explores the five pillars of EDD and demonstrates how MLflow 3.0's new features—including one-line tracing, automated evaluation, human-in-the-loop feedback, and monitoring—provide a comprehensive toolkit to ensure agent quality and reliability.

[Full Workshop] Building Metrics that actually work — David Karam, Pi Labs (fmr Google Search)

[Full Workshop] Building Metrics that actually work — David Karam, Pi Labs (fmr Google Search)

This workshop, led by former Google product directors, introduces a methodology for building reliable and tunable evaluation metrics for LLM applications. It details how to create granular 'scoring systems' that break down complex evaluations into simple, objective signals, and then use these systems for model comparison, prompt optimization, and online reinforcement learning.

← PreviousNext →