Eugene Cheah, CEO of Featherless AI, presents a four-step framework for cutting AI inference costs at scale: measure token spend by team and workload, right-size models through evals, classifiers, and fine-tuning, buy GPU capacity with an understanding of the speed-versus-cost tradeoff, and check whether organizational processes can absorb the extra output.
CoreWeave's Fully Connected 2026 Day 1 keynote centers on the AI loop of run, observe, curate, improve and evaluate, and the launch of CoreWeave Forge. It covers production agent operations, agent observability and RL rollouts, Vera Rubin NVL72 and the Vera CPU, and enterprise evaluation practices from Capital One, Caterpillar and Cognition.
An exploration of the recent wave of AI innovations, focusing on the pervasive theme of efficiency. The episode delves into cost-reduction in new frontier models like Anthropic's Claude Opus 5.5 and OpenAI's GPT-6, the emergence of TypeSafe AI's "System One" models like Jev for high-speed, calibrated decision-making, and the groundbreaking open-source Lunar Foundation Model by NASA and IBM, showcasing AI's diverse applications beyond text generation.
Byung-Gon Chun from FriendliAI details how agentic inference fundamentally differs from traditional chat-based LLM usage, requiring a specialized infrastructure to optimize for end-to-end task completion rather than single request latency. He highlights the economic advantage of open-weight models for agentic workflows and explains FriendliAI's innovative inference stack, featuring prefix caching, advanced KV cache management, cache-aware routing, and agent-aware optimization, which together deliver significant speedups and cost savings for production AI agents.
This panel discussion explores the inflection point of Local AI, driven by advanced models, improved hardware, and a robust ecosystem. Experts discuss how this shift addresses critical concerns around privacy, cost, sovereignty, and resilience, emphasizing the pivotal role of open-source AI and specialized models. They delve into technical optimizations, the evolution from generalized to specialized AI, and the challenges of making local AI accessible and performant for both enterprise and individual users.
Alex Cheema from EXO Labs explores the path to a 100x improvement in the price-performance of running frontier AI models locally. The talk covers full-stack optimization strategies, including kernel fusion for a 30% performance boost, RDMA for scalable tensor parallelism, and a novel approach of splitting prefill and decode phases across heterogeneous hardware (e.g., an RTX GPU and Mac Studios) to significantly speed up large-prompt inference.