Jul 08, 2025

LLMOps for eval-driven development at scale

Mercari's engineering team shares their practical, evaluation-centric approach to LLMOps. Learn how they leverage tiered evaluations, strategic tooling for observability, and rapid iteration to productionize LLM features for over 23 million users, emphasizing that good 'evals' are often more critical than model fine-tuning or RAG.

Jul 03, 2025

Unlocking Unstructured Data with LLMs

Shreya Shankar of UC Berkeley discusses DocETL, a MapReduce-style framework that leverages LLMs to extract, analyze, and structure insights from unstructured enterprise data. The conversation covers practical architecture patterns, the role of non-determinism, strategies for model selection (including fine-tuning and multi-LLM pipelines), and the importance of user experience in this emerging field.