NVIDIA Vera CPU: The Foundation for Agentic AI

The talk opens by arguing that agentic AI has put the CPU back on the critical path of AI factory design, and it frames NVIDIA Vera as a CPU built for that shift. To make the case, the speaker first walks through how data center CPUs evolved.

From the pre-cloud era to the cloud era

In the pre-cloud era, around 2013 and 2014, data center CPUs looked much like client CPUs. The speaker uses a two-axis matrix, with single-threaded core performance on the y-axis and throughput on the x-axis. Intel’s Haswell generation (2014) offered up to 18 cores, the design focus was frequency in the 3 to 3.5 GHz range, and no single customer type dictated CPU design.

The cloud era changed that. Hyperscalers grew on scale economics, and the speaker cites the arithmetic of revenue = volume x ASP: CSP revenue grew about 16x while instance pricing roughly halved, so volume, and with it the demand for vCPUs, exploded. Vendors responded in three ways:

• Core counts climbed about 9x, from 14 cores in Haswell to 60 in Intel’s Emerald Rapids (2024) and 128 in AMD’s Turin.
• Chiplet architectures lowered cost through better yields and higher core densities.
• Workloads (batch processing on one side, latency-sensitive front-end work like search and ad serving on the other) were converged onto a single high-core-count, high-throughput design. Hyperscalers handled latency through scale-out techniques like load balancing, CDNs and failover across thousands of servers.

The cost of this path was single-threaded performance. Per-core performance grew only about 2x over roughly seven years, around 10% year over year. The speaker explains the tradeoffs: prioritizing core count left little room for IPC gains, chiplets impose a memory bandwidth penalty, and latency-sensitive applications were left underserved. The cloud era produced the bottom-right quadrant of the matrix, the scale-out CPU.

Why agents need a different CPU

Agentic AI runs in loops that alternate between GPU inference and CPU work such as tool calls, API calls, SQL queries, compiling code, and opening browsers. These loops can repeat hundreds of times, and each pass touches memory several times. The speaker shows a trace from an NVIDIA coding workflow where the GPU time (green) is followed by CPU time (gray), and that gray span is the optimization target. Compressing it speeds up the whole agentic loop, and more cores alone are not enough; the cores need to be faster.

The speaker lists five things agents ask of CPUs:

• Fast single-threaded performance under full load, so cores stay busy without slowing down.
• Efficiency for power-limited data centers, since tokens are bounded by available power.
• Very fast data processing, because AI runs on large volumes of data.
• Strong protection of models and user data, even from the data center operator.
• Massive IO, so the CPU and GPU can loop together without friction and keep expensive GPUs highly utilized.

The conclusion is that agents need neither the pre-cloud CPU nor the cloud CPU, but a new class the speaker calls the max single-threaded CPU at scale: high per-core speed combined with high throughput. The speaker presents this as a case study in NVIDIA’s extreme codesign philosophy. The team did not set out to win the CPU market; it identified the CPU as a bottleneck in the AI factory and set out to fix it.

Vera’s three pillars

Vera targets the agentic profile through three pillars:

1. Performance per core, from the custom NVIDIA Olympus core, which the speaker says delivers about 2x per-core performance.
2. A fast on-chip fabric, based on the second-generation NVIDIA scalable coherent fabric, with 3.4 TB/s of core-to-core bandwidth, about 3x x86. This matters when agents share context or a knowledge base across cores.
3. Memory bandwidth from a data-center-class LPDDR5X subsystem, delivering 1.2 TB/s, about 3x x86, with roughly 40% lower latency.

Olympus is designed around very high per-core performance and efficient throughput scaling. Its features include:

• Spatial multithreading with two threads per physical core, keeping execution units busy when one thread stalls.
• A wide front end with neural branch prediction and a 10-wide decode.
• A deep out-of-order engine that looks ahead in the instruction queue and works around memory and branch stalls.
• Wide integer, branch and vector units with SVE2 support, plus four load and two store pipelines for memory-level parallelism.
• A large private cache hierarchy with advanced prefetching.

The speaker argues the design is balanced: the front end keeps the core fed, the mid-core exposes parallelism, the execution engine processes it, and the cache keeps data moving.

The fabric is monolithic. All cores sit on a single die and communicate on one plane, with no chiplet boundaries to cross, which the speaker describes as zero chiplet hops. Chiplet designs add a boundary, and therefore latency, at every hop.

On memory, the speaker highlights efficiency as well as bandwidth. The LPDDR5X subsystem draws about 40 W, compared with hundreds of watts for competing subsystems, which yields about 5x better memory bandwidth per watt than x86. Vera reaches 1.1 TB/s, about 90% of its peak, while keeping latency steady and under 200 ms. The speaker argues that x86 hits this memory wall much sooner, and that this combination of fabric, bandwidth and low latency is what makes agentic loops fast.

Customer evidence

The speaker points to customer results over spec benchmarks:

• Perplexity cloned a repository and ran test suites in sandboxes. Vera delivered 1.5x faster job completion and 1.9x faster sandbox startup.
• Deep Infra compared Vera against incumbent environments and reported the fastest orchestration (1.8x), fastest intrinsic code execution spawn, fastest memory bandwidth, and highest base throughput.
• ClickHouse, the open-source analytical database used in observability stacks, was run on ClickBench, where Vera now tops the leaderboard. The quoted reaction was that it is ’the fastest machine we’ve measured so far.'

The speaker says these results extend beyond agents to data processing and EDA, which matter for telemetry, logs, metrics and analytics workloads, where faster queries mean more concurrent users and faster insights.

Vera ships in two form factors: a dense liquid-cooled rack with 256 Vera CPUs and over 22,000 cores, and a standard 2U air-cooled enterprise server with two sockets and 172 cores.

CoreWeave’s perspective

Harsh, director of product at CoreWeave, describes how CoreWeave has seen CPU roles change over the past year. CPUs used to be mostly control planes for orchestrators or data preprocessing units. Now customers running reinforcement learning use CPUs for rollouts, and they ask for CPUs to run agents in isolated, secure sandboxes.

CoreWeave tested Vera on its Kubernetes service and its sandbox offering. Container startup improved by up to 3x. Harsh argues this matters because agent workloads are bursty: thousands of sandboxes are spun up, shut down, and restarted for each task, so startup overhead compounds at scale.

CoreWeave also ran Terminal Bench, a benchmark closer to real agentic usage than traditional CPU suites like SPEC. Vera came out ahead on most tasks, averaging about 1.7x better than comparable x86 numbers. CoreWeave is testing Vera across its stack and plans to offer it through multiple consumption paths.

• Tags:
• Agentic AI
• Feature
• CoreWeave
• AI Factory
• NVIDIA Vera
• CPU Architecture
• NVIDIA Olympus Core
• LPDDR5X Memory
• Chiplet Architecture
• Sandbox Startup
• Terminal Bench
• ClickBench
• Reinforcement Learning Rollouts
• Data Center CPUs