Introduction
On August 24, 2026, NVIDIA published measurements claiming that its Vera Rubin NVL72 system delivers up to 30x higher throughput per megawatt and up to 35x lower cost per million tokens than the GB300 NVL72 on agentic workloads. The company says Vera Rubin is in full production and scaling across the ecosystem.
Two things make the claim worth reading past the headline. The first is what was measured: not chat, but replayed recordings of real AI agent coding sessions. The second is what has not happened yet — NVIDIA measured the results itself, using a benchmark built by SemiAnalysis, and states they are pending SemiAnalysis review. These are vendor-reported figures.
What NVIDIA Actually Measured
The 30x is a single point on a curve, not a blanket rating. NVIDIA's technical post states the number precisely: at 160 output tokens per second per user on the AgentX DeepSeek V4 Pro workload, Vera Rubin NVL72 delivers up to 30x higher throughput per megawatt than GB300 NVL72. AgentX sweeps concurrency to map the trade-off between total throughput and per-user interactivity, so the comparison holds at a chosen responsiveness target rather than across the whole range.
The measured model is DeepSeek V4 Pro alone. NVIDIA does separately name Kimi K3, MiniMax M3, GLM5.3, Qwen3.5 and DeepSeek V4 Pro — but as models where the Blackwell platform leads on AgentX, not as a set of Vera Rubin results. For generational context, NVIDIA puts GB300 NVL72 at up to 15x the throughput per megawatt of H200 NVL8 on DeepSeek V4 Pro 1.6T, and up to 10x lower cost per million tokens; on the larger Kimi K3 2.8T the same comparison reaches roughly 80x.
The choice of denominator is deliberate. NVIDIA's argument is that "for power-constrained AI factories, throughput per megawatt determines AI factory revenue and cost per million tokens determines the profit margin on that revenue." Performance per GPU is the wrong unit when the binding constraint is the electrical feed rather than the rack.
Why Agentic Workloads Need a Different Benchmark
A chat or summarization request is short and self-contained: NVIDIA cites 1K to 8K tokens as typical. An agentic workflow is neither. Context accumulates step by step and can reach hundreds of thousands of input tokens in a single session. NVIDIA cites OpenRouter's State of AI report, drawn from 100 trillion tokens of real usage, showing that average prompt tokens per request grew roughly fourfold and that a single agentic request consumes 15x the tokens of ordinary chat.
That difference is not one of degree, and it is the argument for replaying real agent traces instead of scoring fixed-length prompts. A long, growing context window makes the KV cache — the stored attention state that lets a model avoid recomputing every prior token — the dominant memory consumer, and it grows for the whole session rather than resetting each turn.
AgentX is built for exactly that. It is the agentic-coding benchmark in InferenceX, SemiAnalysis's open-source suite, and it replays prerecorded Claude Code sessions turn by turn using the AIPerf client. The replay preserves each session's context, input and output sequence lengths, reasoning time and tool-call latency, reproducing the KV-cache pressure of the original trajectory. Because every system receives identical recorded traffic, differences should reflect the serving stack rather than tuning for the benchmark. NVIDIA notes that the older fixed 8K-input/1K-output InferenceX scenario has been demoted to maintenance mode as agentic traffic became the leading workload.
The Optimizations Named
NVIDIA credits "extreme codesign across every layer of the platform" rather than silicon alone:
- Disaggregated serving separates prefill from decode so each scales independently, with rate matching synchronizing the rates at which prefill and decode GPUs produce tokens.
- Distributed KV-caching extends cache across the scale-up domain, KV-cache offloading tiers less-active context to host and storage, and KV-aware routing sends requests to GPUs already holding the relevant context.
- Large-scale expert parallelism and fused CUDA kernels such as MegaMoE target Mixture-of-Experts models.
- Enhanced fifth-generation Tensor Cores, a third-generation Transformer Engine, and NVFP4 quantization compressing weights to 4-bit precision.
Power management is a separate lever from the chip: NVIDIA says its DSX MaxLPS technology manages power across GPU, rack and workload levels to provision up to 40% more GPUs within the same megawatt budget — which raises throughput per megawatt without any per-GPU gain at all.
What Is Not Yet Confirmed
Three caveats come from NVIDIA's own posts.
The results are "currently pending SemiAnalysis review." A vendor running a third party's benchmark on its own hardware is a reasonable starting point, not a settled result, and 30x is a large enough multiple that independent confirmation is the difference between a data point and a marketing number.
The measurements "don't yet reflect Vera CPU performance for tool calling." That omission is larger than it sounds: in the full platform, Vera CPUs handle tool execution and KV-cache offload, and tool calls occupy real wall-clock time in agent sessions. The published figure describes token generation more than end-to-end agent latency. NVIDIA frames this as headroom still uncounted.
Finally, the baseline moves. NVIDIA states that "with continuous software optimizations, performance across both Vera Rubin NVL72 and GB300 NVL72 will continue to improve" — a ratio between two actively optimized software stacks is a snapshot, not a fixed property of the hardware. Neither post breaks the 30x into contributions from silicon, serving software, NVFP4 precision or power provisioning, and the 35x cost figure is published without a stated model or interactivity point at all.
Conclusion
The substantive part of NVIDIA's announcement is methodological. Measuring inference hardware on replayed agent traces — long context, preserved tool-call gaps, sub-agents — is a better proxy for what production GPU fleets actually run in 2026 than fixed-length prompts, and throughput per megawatt is the honest denominator for buyers whose data centers are power-limited. Most of these are inference optimization techniques the industry already knows; the claim is about the scale at which the rack sustains them.
The multiples themselves should be held loosely: 30x is one interactivity point on one model, measured by the vendor, pending review, and excluding the tool-calling path that agentic work depends on.
Sources
- NVIDIA — Up to 30x More Work Per Watt: Vera Rubin NVL72 Sets a New Efficiency Standard for AI Agents (Shruti Koparkar, August 24, 2026)
- NVIDIA Technical Blog — Vera Rubin and Blackwell Set a New Standard for Agentic AI Performance per Watt (Jamie Li and Eduardo Alvarez, August 24, 2026)
- SemiAnalysis — AgentX and InferenceX v3
- OpenRouter — DeepSeek V4 adoption and token consumption data