A product and go-to-market lead at NVIDIA examines the economics of AI inference as agentic AI multiplies token demand. The talk covers matching premium versus efficient models to use cases, model routing, open models, sizing token demand, and extreme co-design across mixture-of-experts models, NVLink systems, KV caching, software stacks, and CPUs to maximize tokens per watt and minimize cost per token.
A session on alternative compute covering the argument for abandoning backpropagation in favor of zero-order optimization and sharded training, optical computing that performs diffusion-style image generation with propagating light, the state of neuromorphic computing in 2026, and how cultured human neurons learned to play Doom through reinforcement learning.
This episode of Mixture of Experts delves into the latest frontier AI news, covering Anthropic's new Fable and Mythos models with their nuanced performance and commercial strategy, OpenAI's alarming security incident involving highly capable AI agents, Runway's innovative Solaris introducing 'Interface World Models' for dynamic UI generation, and Anthropic's Model Hardware Standard pushing AI into physical equipment control.
Dylan Patel, founder of SemiAnalysis, details how software-hardware co-design drives 100x gains in AI performance, citing examples like DeepSeek's optimization for Nvidia Hopper and the architectural differences between OpenAI's sparse and Anthropic's dense models. He introduces InferenceX, a living benchmark tracking a 60x annual cost reduction in AI quality, and explains why the "CUDA moat" narrative is shifting. Patel also delves into the persistent compute crunch, the strategic rise of neoclouds, and Jensen Huang's efforts to foster a multipolar AI world to prevent hyperscaler dominance.