Terminal bench

Don't Trust Your AI Agents. Test Them Like a Scientist — Stephanie Jarmak, Omni Analytics

Don't Trust Your AI Agents. Test Them Like a Scientist — Stephanie Jarmak, Omni Analytics

Stephanie Jarmak, who moved from astronomy into building coding benchmarks and agent evals, argues that most agent evaluations are easy to misread. She covers how harness choice can swamp model differences, why public benchmarks serve shortlisting rather than decisions, how to design experiment-style evals with realistic tasks, repeated runs, and full traces, and how to set quality gates, manage human review load, verify agent records, and treat production as the final test.

How Cline Embraces an Open-Weights Model Ecosystem for AI Agents

How Cline Embraces an Open-Weights Model Ecosystem for AI Agents

Cline's CEO explains why open-weight models are rapidly taking over coding-agent workloads, and why the agent harness, not raw model capability, determines reliability. The talk covers model-specific tuning and failure handling, context and token optimization, self-improving harness loops, an SDK-first and UI-agnostic architecture, interactive evaluation built from real-world usage data, and production inference challenges such as KV cache hit rates.

NVIDIA Vera CPU: The Foundation for Agentic AI

NVIDIA Vera CPU: The Foundation for Agentic AI

NVIDIA presents Vera, a CPU designed for agentic AI workloads such as tool calls, code execution, sandboxes and reinforcement learning loops. The talk traces how cloud economics shaped CPU design toward high core counts at the expense of per-core speed, then explains how Vera's Olympus core, monolithic fabric and LPDDR5X memory deliver high single-threaded performance at scale. CoreWeave then shares its early results running Vera for sandboxes and agent workloads.