Aug 07, 2026

Compression at the Edge — Chris Alexiuk, NVIDIA

This panel discussion explores the critical role of model compression, particularly quantization, in democratizing AI. It delves into how massive models like GLM 5.2 can be shrunk by over 80% without equivalent performance loss, thanks to techniques like mixed-precision quantization and understanding uneven layer importance. The discussion covers NVIDIA's NVFP4 format, challenges posed by new model architectures, the preference for KL divergence over accuracy benchmarks, and the vision of future AI running efficiently on all local devices.

Jul 28, 2026

Llama.cpp vs vLLM: Which Local LLM Engine Actually Scales?

Explore the strengths and optimal use cases of `llama.cpp` and `vLLM` for local LLM inference. `llama.cpp` excels on consumer hardware with optimizations like quantization and CPU support, while `vLLM` is designed for production-scale efficiency with features like continuous batching and speculative decoding on high-performance accelerators.

Jul 25, 2026

Why Large? Tiny LMs & Agents on Edge/Robotics — Cormac Brick, Google

Cormac Brick from Google AI Edge discusses how the increasing constraint of DRAM cost on edge devices necessitates the development and deployment of increasingly smaller AI models. He outlines the work of his team in optimizing models like Gemma, achieving 2.9 bits per weight for a 2 billion parameter model capable of running on a Raspberry Pi at 7.6 tokens/second, or on an NPU at 31 tokens/second decode for vision tasks. The talk delves into 'tiny models' (50M-500M parameters) that extend AI to older devices and enable features like robust voice-to-function calling via fine-tuning with synthetic data, exemplified by an offline voice dictation app.

Jun 28, 2026

Turbocharge Your Agent's Retrieval with TurboQuant - Shashi Jagtap, Superagentic AI

This talk introduces TurboQuant, a training-free compression method from Google Research that reduces embedding memory footprint by 5x (from 32-bit to 3-4 bits) without losing search quality. It details how TurboQuant works through scalar quantization and a crucial one-bit error correction step, QJL, enabling agents to remember more on existing hardware by optimizing both KV cache and RAG vector stores. A live demo showcases its effectiveness, making it a vendor-neutral solution for efficient AI agent retrieval.

Jun 16, 2026

You Might Not Need 50 Diffusion Steps — Ziv Ilan, Nvidia

Ziv Ilan from NVIDIA details how latency in video diffusion models can be drastically reduced to achieve real-time generation. He presents a layered approach combining dynamic quantization for memory and speed, chunk-based caching to skip redundant denoising computations, and, most critically, step distillation—training models to achieve high-quality output in significantly fewer steps. These techniques, packaged in the open-source FastGen repository, offer additive performance gains, enabling real-time video on a single Blackwell B200 GPU.

May 20, 2026

The Future of AI – Key Trends Shaping What’s Next • Ekaterina Sirazitdinova • YOW! 2025

Ekaterina Sirazitdinova from NVIDIA provides a high-level overview of the latest trends shaping the future of AI, covering the evolution from early deep learning to the rise of agentic and physical AI, and diving deep into the critical optimization techniques required to deploy these powerful models efficiently.