NVIDIA claims up to 30x more throughput per megawatt for AI agents on Vera Rubin NVL72

NVIDIA is pushing a new performance benchmark for the next generation of AI workloads, arguing that agentic systems, which take multi-step actions and grow their context over time, need radically different efficiency metrics than chatbots. In a blog post published Monday, the company said its Vera Rubin NVL72 platform delivers up to 30 times higher throughput per megawatt than NVIDIA’s GB300 NVL72, on a set of measured agent workloads tied to real-world agent coding sessions.

Agentic AI refers to AI systems that do more than answer prompts. Instead of producing a single response, an agent typically plans steps, calls tools, spawns sub-agents for specialized tasks, and keeps reasoning until a job is complete. NVIDIA highlighted the mechanics behind why these systems can be expensive: an agent that researches a company for an investment decision may query financial databases, search news and filings, run peer comparisons and valuation checks through sub-agents, then synthesize results. NVIDIA’s framing is that each step adds to the growing “input” context for subsequent steps, so token use accumulates across the session rather than ending at the first model completion.

The post ties this behavior to token demand. NVIDIA cited OpenRouter data that agentic workloads consume about 15 times more tokens than a simple chat request. While the blog describes the trend conceptually, NVIDIA’s more concrete claim is about measured inference throughput under power constraints. The company said its measured results show Vera Rubin NVL72 systems deliver up to 30x higher throughput per megawatt than GB300 NVL72 when running agentic workloads.

To ground the comparison, NVIDIA said it used SemiAnalysis AgentX, a workload intended to preserve features of real agent runs. NVIDIA described the AgentX workload as recorded real-world agentic coding sessions, keeping actual context growth, tool calls, and sub-agent spawning intact. That choice matters in NVIDIA’s view because agent performance cannot be assessed as a single inference request. NVIDIA said agent sessions can accumulate hundreds of thousands of input tokens, with wide variability in input and output lengths, so evaluation must capture the full workflow rather than isolated model calls.

Within the same post, NVIDIA offered an additional comparison point involving its previous-generation architecture. It said GB300 NVL72 can deliver up to 15x better throughput per megawatt than the NVIDIA Hopper architecture on the DeepSeek V4 Pro model. NVIDIA then characterized Vera Rubin as extending the advantage across what it calls the “Pareto curve,” stating Vera Rubin can lift performance to as much as 30x higher throughput per megawatt compared with GB300 NVL72, again on DeepSeek V4 Pro. NVIDIA said these results are early and are currently pending SemiAnalysis review, so readers are asked to treat them as preliminary.

Efficiency is also connected to cost per token, the amount charged for generating a unit of model output in many deployments. NVIDIA said throughput per megawatt can translate directly into token economics and claimed that Vera Rubin NVL72 can produce tokens at up to 35x lower cost per million tokens than GB300 NVL72. The company also said NVIDIA DSX MaxLPS manages power across the GPU, rack, and workload levels to provision up to 40% more GPUs within the same megawatt budget, which it said pushes throughput per megawatt higher at AI factory scale.

The Vera Rubin performance claim is supported by hardware and software details NVIDIA says are aimed at reducing the total computation per output token. NVIDIA pointed to enhanced fifth-generation Tensor Cores and the third-generation Transformer Engine to accelerate both “prefill” and “decode” stages of inference. Prefill is the phase where the model processes the prompt and builds internal context; decode is the phase where it generates tokens step-by-step. NVIDIA also mentioned NVFP4 quantization, which compresses model weights to 4-bit precision to reduce the memory footprint and increase throughput while aiming not to sacrifice output quality. Separately, NVIDIA said its NVLink interconnect and NVLink Switches, in what it described as sixth generation form, provide high-bandwidth, low-latency communication across GPUs, which NVIDIA said is essential for techniques like large-scale expert parallelism and distributed KV-caching. KV-caching is a method used in transformer models to reuse intermediate attention data so the system does not repeatedly recompute it during long-context generation.

NVIDIA also described the broader platform as more than just GPUs. It called Vera Rubin a seven-chip architecture that includes the Vera CPU, Groq 3 LPU, and networking and acceleration components such as NVLink 6 Switch, BlueField-4 DPU, Spectrum-6 SPX, and ConnectX-9 SuperNIC. The company added that Vera Rubin is in full production and scaling across its ecosystem, while also saying its software stack is codesigned with the hardware. NVIDIA referenced CUDA kernel optimizations, inference runtimes including TensorRT LLM, and serving frameworks such as NVIDIA Dynamo.

What is still unclear is how far the headline gains will generalize across models, agent designs, and customer configurations. NVIDIA’s blog ties its measured numbers to the SemiAnalysis AgentX workload, described as recorded coding sessions, and it highlights DeepSeek V4 Pro in specific throughput-per-megawatt examples. The company also said the results do not yet reflect Vera CPU performance for tool calling, which is a key part of many agent workflows. Until SemiAnalysis’ review is completed, the market will likely watch for whether independent assessment confirms NVIDIA’s up to 30x figure and whether the approach holds on other agent tasks beyond coding sessions.

Going forward, the immediate test for NVIDIA’s claim will be whether AI customers treat “throughput per megawatt” and “cost per million tokens” as central procurement criteria for agentic systems, and whether software and infrastructure providers can translate these measured gains into stable production performance. NVIDIA said it plans continuous software optimizations to improve performance across Vera Rubin and GB300 NVL72, so the next announcement to watch is whether the delta narrows or widens after further tuning and after independent benchmarking finishes.