Oct 10, 2026

From 15% to 90% GPU Utilization: Fix the Data Pipeline, Not the Model

Tarun Sunkaraneni from Amazon AGI shows how a multimodal SFT pipeline that spent about 85% of its time waiting on data was brought to roughly 90% GPU utilization. He walks through concurrency with asyncio and Ray actors, prefetching, zero-copy transfer via Ray's object store, and a network-card bottleneck that appears only at scale, and argues that the hardest part of training throughput is often keeping GPUs fed rather than optimizing the model.

Oct 09, 2026

Data pipeline at Petabyte scale: Production Video Curation with Ray on CoreWeave

A walkthrough of how Anyscale and CoreWeave scaled a video captioning pipeline from 600 GB to about 6 PB, running managed Ray on CoreWeave Kubernetes Service with CAIOS caching. The talk covers the stack, the bottlenecks that drove GPU utilization from about 10% to near 100%, storage and fault-tolerance design, and the throughput achieved across 1,500 or more GPUs.

Oct 06, 2026

Is Speculative Decoding Worth It? Profiling vLLM on NVIDIA Blackwell — Akamai

Sheilah Kirui explains speculative decoding for LLMs, detailing how a small draft model proposes tokens for a larger target model to verify in a single pass, accelerating the decoding phase. She covers the memory costs of hosting two models, criteria for choosing an effective draft model, and demonstrates its benefits for structured output (1.6x faster with high acceptance) versus its limited gains for creative tasks. The talk concludes by outlining when speculative decoding is most advantageous, considering VRAM availability, batch size, workload type, and context length.

Sep 19, 2026

Large clusters for small models — Daniel Svonava, Superlinked

Daniel Svonava argues that small open-source models, when specialized and properly served, can achieve frontier performance for specific tasks, offering significant cost savings and latency improvements over large, generalized models or managed endpoints. He highlights challenges with existing serving infrastructure for a fleet of diverse small models and introduces Superlinked's open-source solution, which leverages a pull-based, centralized queuing system, a Rust sidecar for runtime abstraction, and an autoresearch loop for automated model tuning, enabling high throughput (e.g., half a million embedding tokens/sec on one GPU) and efficient deployment of specialized models and LoRAs.

Aug 10, 2026

Taking Reinforcement Learning Cross Datacenter — Nan Jiang, Modal

Nan Jiang introduces "Adam absorption" to revolutionize RL model synchronization. By exploiting finite precision serving and small Adam steps, less than 1% of served model weights actually change, allowing for 500MB patches instead of 500GB checkpoints. This enables a distributed "bulletin board" architecture, decoupling trainers from global rollout fleets and unlocking elastic, cross-region GPU capacity for RL.

Jun 18, 2026

The AI Frontier: from FLOPs to Megawatts — Anjney Midha, AMP

Anjney Midha unpacks the critical bottlenecks in AI scaling beyond just GPU acquisition, advocating for responsible infrastructure, community-aligned data centers, and an independent system operator model for compute. He discusses the perils of research hoarding, the rise of researcher CEOs, and how Anthropic's culture of "preparedness" and "output maxing" led to its success, while also highlighting his personal mission to use AI for precise end-of-life prediction.