Ollama Co-founder Jeffrey Morgan discusses the enterprise shift to open models, driven by cost, customization, and coding agents, leading to a 150X token growth. He highlights the dominance of Chinese models in cloud consumption, the hardware renaissance enabling local LLMs, and the rise of ultra-efficient "flash" models. Morgan also shares Ollama's journey, from its YC pivot to its monetization strategy, and reflects on lessons for building AI companies.
Cerebras CTO Sean Lie discusses the
A deep dive into the current AI boom, exploring why demand for intelligence is likely underestimated and the outcome won't necessarily be winner-take-all. The discussion covers the rapid economics of AI infrastructure, the risk of an AI shortage over a bubble, the potential for orbital compute and re-industrializing America, and NVIDIA's central role as the "central bank of AI" in a multi-model future.
Steve Hou, Head of Research at Silicon Data, unpacks the data behind the AI compute market, revealing persistent tightness in GPU rentals and rising residual values, despite talk of oversupply. He discusses the methodologies behind their GPU price indices and forward curves, the evolving hardware landscape beyond Nvidia (including AMD, Cerebras, and TPUs), and trends in LLM token economics. The conversation also delves into the complex financial aspects of AI data center buildouts, the rise of specialized models, China's emerging AI hardware ecosystem, and the growing importance of power constraints and distributed AI infrastructure.
This panel discussion explores the critical role of model compression, particularly quantization, in democratizing AI. It delves into how massive models like GLM 5.2 can be shrunk by over 80% without equivalent performance loss, thanks to techniques like mixed-precision quantization and understanding uneven layer importance. The discussion covers NVIDIA's NVFP4 format, challenges posed by new model architectures, the preference for KL divergence over accuracy benchmarks, and the vision of future AI running efficiently on all local devices.
A deep dive into model routing strategies for AI/ML production, featuring experts from Cognition, OpenRouter, and NVIDIA. Key topics include optimizing costs with multi-model systems, delegating tasks between frontier and smaller models, managing context efficiently (sidekicks, compaction), and adapting to dynamic task complexities. The panel discusses the fragility of naive routing, the cost implications of in-distribution vs. out-of-distribution tasks, and the evolution of auto-routers driven by real-world usage patterns like OpenClaw's heartbeats. Insights also cover NVIDIA's Flex Run for dynamic model sizing, hallucination probes for detecting model limitations, and the future of hybrid local/cloud routing and model collaboration.