Kubernetes

Paving the Road to Production: Governed Enterprise AI Agents — Harish Gaggar, Credit Karma

Paving the Road to Production: Governed Enterprise AI Agents — Harish Gaggar, Credit Karma

Harish Gaggar of Intuit Credit Karma walks through an internal paved path for taking enterprise AI agents from experiments to production. The talk covers portal-based registration and provisioning, a centralized authenticated MCP gateway with custom reviewed MCP servers, gateway monitoring with OpenTelemetry, Tempo, Prometheus, and Grafana, LangGraph workflows with policy checks and human approval, durable memory through threads and checkpoints, agent tracking and evaluation records, PII filtering, and context optimization for token reduction.

Data pipeline at Petabyte scale: Production Video Curation with Ray on CoreWeave

Data pipeline at Petabyte scale: Production Video Curation with Ray on CoreWeave

A walkthrough of how Anyscale and CoreWeave scaled a video captioning pipeline from 600 GB to about 6 PB, running managed Ray on CoreWeave Kubernetes Service with CAIOS caching. The talk covers the stack, the bottlenecks that drove GPU utilization from about 10% to near 100%, storage and fault-tolerance design, and the throughput achieved across 1,500 or more GPUs.

Quota Isn't Enough: Topology Aware Scheduling on Modern GPU Clusters

Quota Isn't Enough: Topology Aware Scheduling on Modern GPU Clusters

A deep dive into why rack-scale NVL72 GPU hardware makes topology a scheduling requirement, and how Kueue's Topology Aware Scheduling (TAS) brings NVLink domains, rack boundaries, and PodSet slices into Kubernetes admission. Covers preferred vs. required placement, the performance cliff between NVLink and InfiniBand, and production lessons on sticky scheduling, quota debugging, fragmentation, node hot swaps, and the upstream path toward configurable preemption.

Dedicated Inference

Dedicated Inference

A walkthrough of CoreWeave's three inference paths (Serverless, Dedicated, and Inference on CKS), covering when each fits, how a multi-agent research assistant uses OpenAI-compatible endpoints across model types, and live demos of gateways, deployments, and Kubernetes-based vLLM, Dynamo, and LLM-D serving.

Essential Skills for Becoming an AI Engineer: RAG, AI Agents, & More

Essential Skills for Becoming an AI Engineer: RAG, AI Agents, & More

Cedric Clyburn outlines the essential skills for AI engineering, distinguishing it from ML research. He details a three-tier skill stack covering foundational programming (Python, Git, APIs), AI-specific techniques (embeddings, RAG, AI agents), and deployment strategies (containerization, observability). The discussion emphasizes judgment over code generation and highlights key use cases like RAG systems and agent-based solutions.

Platform Engineering for Developers, Architects & the Rest of Us • Daniel Bryant • GOTO 2025

Platform Engineering for Developers, Architects & the Rest of Us • Daniel Bryant • GOTO 2025

Daniel Bryant discusses platform engineering for software developers and architects, emphasizing treating platforms as internal products with developers as customers. He outlines a three-layered architecture, the evolution from monolithic systems to microservices, and the importance of 'golden bricks' over 'golden paths' for composability. Key takeaways include API-first design, minimizing cognitive load, avoiding leaky abstractions, and measuring success through frameworks like DORA and DevEx to achieve speed, safety, and scale.