Sep 19, 2026

Large clusters for small models — Daniel Svonava, Superlinked

Daniel Svonava argues that small open-source models, when specialized and properly served, can achieve frontier performance for specific tasks, offering significant cost savings and latency improvements over large, generalized models or managed endpoints. He highlights challenges with existing serving infrastructure for a fleet of diverse small models and introduces Superlinked's open-source solution, which leverages a pull-based, centralized queuing system, a Rust sidecar for runtime abstraction, and an autoresearch loop for automated model tuning, enabling high throughput (e.g., half a million embedding tokens/sec on one GPU) and efficient deployment of specialized models and LoRAs.

Sep 19, 2026

What's New in Inference Engineering — Philip Kiely, Baseten

Philip Kiely provides an update on recent advancements in inference engineering, covering new techniques in quantization (TurboQuant's specific application), KV cache compaction (introducing 'Still' for learned, compressed memory), and speculative decoding (from Eagle 3 to Dlash and DSpark, including continuous retraining). He emphasizes the blurring lines between training and inference and differentiates between local and data center optimization strategies.

Sep 19, 2026

Routing LLM Inference in Production: From Engine Signals to Policy — Qianru Lao & Lu Zhang, OpenAI

This talk outlines OpenAI's journey in building a robust inference load balancer. It details the transition from an initial feedback-loop-driven system, which suffered from explainability issues and disruptive oscillations, to a sophisticated control plane/data plane architecture. The new system leverages a global view to optimize end-to-end latency across all traffic, balancing network distance with engine-side queuing, and includes critical protection mechanisms like outlier penalties, dynamic retry budgets, and load shedding to ensure stability in production.

Sep 19, 2026

Operating Distributed Inference Systems at Scale — Nishant Gupta & Naman Ahuja, Meta

Meta's Nishant Gupta and Naman Ahuja explain how AI inference, the company's fastest-growing workload, requires a dedicated orchestration layer akin to Kubernetes for virtual machines. They detail the unique challenges of LLM serving compared to traditional microservices, emphasizing the intricate coupling of infrastructure layers, the distributed transaction nature of requests, and a seven-axis framework for scheduling. The talk concludes with the imperative to optimize for "cost per successful task" and the necessity of an inference-specific control plane to manage the complex interplay of latency, cost, and throughput.

Sep 18, 2026

A Worm With 302 Neurons Inspired Their Architecture — Ramin Hasani, Liquid AI

Liquid AI began by drawing inspiration from the C. elegans worm to develop Liquid Neural Networks, focusing on continuous-time, differentiable systems for efficient intelligence. Facing scaling challenges, they developed STAR, a meta-AI system for architectural search, optimizing hybrid models for quality, memory, latency, and speed, leading to their Liquid Foundation Models (LFMs). Their core focus is on bringing AI to the edge (on-device, in-car) for cost reduction, privacy, and offline capabilities, with successful deployments at Mercedes-Benz and Shopify. Liquid is also building a platform for 'sovereign AI,' enabling enterprises to customize and own their intelligence end-to-end, and researching massively multimodal systems, efficiency across the AI stack, and novel data types like DNA.

Sep 18, 2026

Databricks CEO: Stop Scaring People About AI

Databricks CEO Ali Ghodsi discusses the current state of AI, arguing against existential risk alarmism and advocating for a focus on immediate threats like AI-powered cyberattacks. He provides a rigorous definition for recursive self-improvement and explains how building an organizational 'ontology' can unlock massive productivity gains in enterprises. Ghodsi also covers the shift to multi-model AI strategies and cost management, highlighting how infrastructure is evolving to support agentic systems.