Aug 27, 2026

Can LLMs Write Fast Multi-GPU Kernels? — Simran Arora, Together AI

Simran Arora discusses the critical bottleneck shift in large AI workloads from GPU compute to inter-GPU communication. Her team's solution, ParallelKittens, offers a set of primitives to optimize multi-GPU kernels by leveraging fundamental transfer mechanisms and compute-communication overlapping. They introduce ParallelKernelBench, a benchmark to evaluate AI models' ability to generate such kernels, revealing that while models can handle syntax, they struggle with deeper reasoning about communication patterns and hardware trade-offs.

Aug 18, 2026

Infra behind Krea 2: How to train and serve at scale — Gabriel Jorge Menezes, Krea.ai

Krea's Gabriel Jorge Menezes shares critical insights into building the infrastructure for large-scale ML model training and serving. Key takeaways include the necessity of custom metrics beyond standard GPU utilization, aggressive checkpointing on ultra-fast storage to counter frequent cluster crashes, and a dynamic Kubernetes-based system using gang scheduling, virtual-kubelet, and taints to seamlessly shift inference workloads to external providers when training consumes on-prem GPUs. The approach highlights practical solutions for silent failures, thermal management, and optimizing resource utilization in a unified production and training environment.