Oct 13, 2026

Judge a Transaction by the Company It Keeps — Somya Rai & MD Sharique, EXL

EXL developed an advanced fraud detection system that augments XGBoost with Graph Neural Network (GNN) embeddings. Traditional tabular models struggle to capture fraud signals residing in transaction neighborhoods. The system's innovative design includes meticulous sampling strategies, disjoint monthly graph construction for leakage control, and a GraphSAGE architecture that aggregates mean, max, and standard deviation of neighbor features to detect subtle fraud patterns. Emphasizing explainability for regulatory compliance and analyst trust, it also features a highly optimized serving architecture achieving 3ms P99 latency, pivoting to edge prediction to reduce cache footprint from 60GB to 2-3GB. This led to a 27-33% relative lift in true positive rates and a projected $65 million incremental fraud capture.

Oct 10, 2026

From 15% to 90% GPU Utilization: Fix the Data Pipeline, Not the Model

Tarun Sunkaraneni from Amazon AGI shows how a multimodal SFT pipeline that spent about 85% of its time waiting on data was brought to roughly 90% GPU utilization. He walks through concurrency with asyncio and Ray actors, prefetching, zero-copy transfer via Ray's object store, and a network-card bottleneck that appears only at scale, and argues that the hardest part of training throughput is often keeping GPUs fed rather than optimizing the model.

Oct 10, 2026

Small Models, Big Results: Training a Finance Agent for Under $500 — Charles Dickens, Snorkel AI

Snorkel AI and UC Berkeley's Sky Computing Lab show that a 4B Qwen3 model, trained with reinforcement learning for under $500 using the rLLM framework, beats its 235B sibling on expert-validated financial question answering built from SEC 10-K filings. The talk covers the FinQA dataset construction and verification, common failure modes of frontier models, transfer to multi-table and general tool-use tasks, and ablations showing that simple data and binary rewards outperform more elaborate setups, since the real bottleneck is disciplined tool use rather than reasoning depth.

Oct 09, 2026

Data pipeline at Petabyte scale: Production Video Curation with Ray on CoreWeave

A walkthrough of how Anyscale and CoreWeave scaled a video captioning pipeline from 600 GB to about 6 PB, running managed Ray on CoreWeave Kubernetes Service with CAIOS caching. The talk covers the stack, the bottlenecks that drove GPU utilization from about 10% to near 100%, storage and fault-tolerance design, and the throughput achieved across 1,500 or more GPUs.

Oct 06, 2026

Is Speculative Decoding Worth It? Profiling vLLM on NVIDIA Blackwell — Akamai

Sheilah Kirui explains speculative decoding for LLMs, detailing how a small draft model proposes tokens for a larger target model to verify in a single pass, accelerating the decoding phase. She covers the memory costs of hosting two models, criteria for choosing an effective draft model, and demonstrates its benefits for structured output (1.6x faster with high acceptance) versus its limited gains for creative tasks. The talk concludes by outlining when speculative decoding is most advantageous, considering VRAM availability, batch size, workload type, and context length.

Oct 06, 2026

What Is an Inference Engine, Anyway? — Charles Frye, Modal

Charles Frye explains the architecture, performance engineering, and operational challenges of LLM inference engines. He dissects workload types, core components like schedulers and kernel backends, and advanced techniques such as KV caching, CUDA graphs, and speculative decoding, concluding with essential strategies for observability and debugging in production environments.