Eugene Cheah, CEO of Featherless AI, presents a four-step framework for cutting AI inference costs at scale: measure token spend by team and workload, right-size models through evals, classifiers, and fine-tuning, buy GPU capacity with an understanding of the speed-versus-cost tradeoff, and check whether organizational processes can absorb the extra output.
Cline's CEO explains why open-weight models are rapidly taking over coding-agent workloads, and why the agent harness, not raw model capability, determines reliability. The talk covers model-specific tuning and failure handling, context and token optimization, self-improving harness loops, an SDK-first and UI-agnostic architecture, interactive evaluation built from real-world usage data, and production inference challenges such as KV cache hit rates.
A product and go-to-market lead at NVIDIA examines the economics of AI inference as agentic AI multiplies token demand. The talk covers matching premium versus efficient models to use cases, model routing, open models, sizing token demand, and extreme co-design across mixture-of-experts models, NVLink systems, KV caching, software stacks, and CPUs to maximize tokens per watt and minimize cost per token.
IBM's Mixture of Experts panel examines OpenAI's decision to withhold GPT-6.1 Astra, Anthropic's Sonnet 5.5 blurring the line between mid-tier and top-tier models, and Meta's Muse agent as a bridge between consumer AI habits and enterprise expectations, with a detour into token budgets, model routing, and AI governance.
Siraj Gupta of Warp outlines three practical ways software factories can improve over time: self-updating skills driven by outer-loop agents with pull-request review, versioned and traceable persistent memory that works across harnesses, and cost-aware model routing with custom rules and evals.
Ignacio Martinez's workshop delves into agent harness engineering, emphasizing that while LLMs are a 'frozen' reasoning core, the surrounding harness is where control and reliability are established. He outlines a seven-layer architecture for building robust agent harnesses, covering critical aspects like hybrid storage solutions, advanced memory engineering (including in-database embeddings and context card management), the semantic layer (Umwelt), agent loops, and context engineering. The session also explores continual learning through 'skill promotion' and addresses practical challenges like managing vast tool sets and optimizing model interactions.