---
source: 'https://howaiworks.ai/blog/nvidia-vera-rubin-nvl72-agentic-efficiency'
section: blog
title: NVIDIA Claims 30x Efficiency Gain for AI Agents on Vera Rubin
description: >-
  NVIDIA reports Vera Rubin NVL72 delivers up to 30x higher throughput per
  megawatt and 35x lower token cost than GB300 NVL72 on agentic coding
  workloads.
date: '2026-08-25'
author: HowAIWorks Team
tags:
  - NVIDIA
  - Vera Rubin
  - AI Agents
  - AI Infrastructure
  - Inference
  - AI Hardware
  - Efficiency
  - DeepSeek
  - AI News
newsSource: NVIDIA
breakingNews: false
readingTime: 5 minutes
isReviewed: false
---

# NVIDIA Claims 30x Efficiency Gain for AI Agents on Vera Rubin

> NVIDIA reports Vera Rubin NVL72 delivers up to 30x higher throughput per megawatt and 35x lower token cost than GB300 NVL72 on agentic coding workloads.

## Introduction

On August 24, 2026, NVIDIA published measurements claiming that its Vera Rubin NVL72 system delivers **up to 30x higher throughput per megawatt** and **up to 35x lower cost per million tokens** than the GB300 NVL72 on agentic workloads. The company says Vera Rubin is in full production and scaling across the ecosystem.

Two things make the claim worth reading past the headline. The first is what was measured: not chat, but replayed recordings of real [AI agent](https://howaiworks.ai/glossary/ai-agent) coding sessions. The second is what has not happened yet — NVIDIA measured the results itself, using a benchmark built by SemiAnalysis, and states they are pending SemiAnalysis review. These are vendor-reported figures.

## What NVIDIA Actually Measured

The 30x is a single point on a curve, not a blanket rating. NVIDIA's technical post states the number precisely: at **160 output tokens per second per user** on the AgentX DeepSeek V4 Pro workload, Vera Rubin NVL72 delivers up to 30x higher throughput per megawatt than GB300 NVL72. AgentX sweeps concurrency to map the trade-off between total throughput and per-user interactivity, so the comparison holds at a chosen responsiveness target rather than across the whole range.

The measured model is [DeepSeek V4 Pro](https://howaiworks.ai/models/deepseek) alone. NVIDIA does separately name Kimi K3, MiniMax M3, GLM5.3, Qwen3.5 and DeepSeek V4 Pro — but as models where the *Blackwell* platform leads on AgentX, not as a set of Vera Rubin results. For generational context, NVIDIA puts GB300 NVL72 at up to 15x the throughput per megawatt of H200 NVL8 on DeepSeek V4 Pro 1.6T, and up to 10x lower cost per million tokens; on the larger Kimi K3 2.8T the same comparison reaches roughly 80x.

The choice of denominator is deliberate. NVIDIA's argument is that "for power-constrained AI factories, throughput per megawatt determines AI factory revenue and cost per million tokens determines the profit margin on that revenue." Performance per GPU is the wrong unit when the binding constraint is the electrical feed rather than the rack.

## Why Agentic Workloads Need a Different Benchmark

A chat or summarization request is short and self-contained: NVIDIA cites 1K to 8K tokens as typical. An [agentic workflow](https://howaiworks.ai/glossary/agentic-workflow) is neither. Context accumulates step by step and can reach hundreds of thousands of input [tokens](https://howaiworks.ai/glossary/token) in a single session. NVIDIA cites OpenRouter's State of AI report, drawn from 100 trillion tokens of real usage, showing that average prompt tokens per request grew roughly fourfold and that a single agentic request consumes **15x the tokens** of ordinary chat.

That difference is not one of degree, and it is the argument for replaying real agent traces instead of scoring fixed-length prompts. A long, growing [context window](https://howaiworks.ai/glossary/context-window) makes the [KV cache](https://howaiworks.ai/glossary/kv-cache) — the stored attention state that lets a model avoid recomputing every prior token — the dominant memory consumer, and it grows for the whole session rather than resetting each turn.

AgentX is built for exactly that. It is the agentic-coding benchmark in **InferenceX**, SemiAnalysis's open-source suite, and it replays prerecorded **Claude Code** sessions turn by turn using the AIPerf client. The replay preserves each session's context, input and output sequence lengths, reasoning time and tool-call latency, reproducing the KV-cache pressure of the original trajectory. Because every system receives identical recorded traffic, differences should reflect the serving stack rather than tuning for the benchmark. NVIDIA notes that the older fixed 8K-input/1K-output InferenceX scenario has been demoted to maintenance mode as agentic traffic became the leading workload.

## The Optimizations Named

NVIDIA credits "extreme codesign across every layer of the platform" rather than silicon alone:

- **[Disaggregated serving](https://howaiworks.ai/glossary/disaggregated-serving)** separates prefill from decode so each scales independently, with **rate matching** synchronizing the rates at which prefill and decode GPUs produce tokens.
- **Distributed KV-caching** extends cache across the scale-up domain, **KV-cache offloading** tiers less-active context to host and storage, and **KV-aware routing** sends requests to GPUs already holding the relevant context.
- **Large-scale expert parallelism** and **fused CUDA kernels** such as MegaMoE target [Mixture-of-Experts](https://howaiworks.ai/glossary/mixture-of-experts) models.
- **Enhanced fifth-generation Tensor Cores**, a third-generation Transformer Engine, and **NVFP4 [quantization](https://howaiworks.ai/glossary/quantization)** compressing weights to 4-bit precision.

Power management is a separate lever from the chip: NVIDIA says its DSX MaxLPS technology manages power across GPU, rack and workload levels to provision **up to 40% more GPUs within the same megawatt budget** — which raises throughput per megawatt without any per-GPU gain at all.

## What Is Not Yet Confirmed

Three caveats come from NVIDIA's own posts.

The results are "currently pending SemiAnalysis review." A vendor running a third party's benchmark on its own hardware is a reasonable starting point, not a settled result, and 30x is a large enough multiple that independent confirmation is the difference between a data point and a marketing number.

The measurements "don't yet reflect Vera CPU performance for tool calling." That omission is larger than it sounds: in the full platform, Vera CPUs handle tool execution and KV-cache offload, and tool calls occupy real wall-clock time in agent sessions. The published figure describes token generation more than end-to-end agent latency. NVIDIA frames this as headroom still uncounted.

Finally, the baseline moves. NVIDIA states that "with continuous software optimizations, performance across both Vera Rubin NVL72 and GB300 NVL72 will continue to improve" — a ratio between two actively optimized software stacks is a snapshot, not a fixed property of the hardware. Neither post breaks the 30x into contributions from silicon, serving software, NVFP4 precision or power provisioning, and the 35x cost figure is published without a stated model or interactivity point at all.

## Conclusion

The substantive part of NVIDIA's announcement is methodological. Measuring inference hardware on replayed agent traces — long context, preserved tool-call gaps, sub-agents — is a better proxy for what production [GPU](https://howaiworks.ai/glossary/nvidia-gpu-ai) fleets actually run in 2026 than fixed-length prompts, and throughput per megawatt is the honest denominator for buyers whose data centers are power-limited. Most of these are [inference optimization](https://howaiworks.ai/glossary/inference-optimization) techniques the industry already knows; the claim is about the scale at which the rack sustains them.

The multiples themselves should be held loosely: 30x is one interactivity point on one model, measured by the vendor, pending review, and excluding the tool-calling path that agentic work depends on.

## Sources

- [NVIDIA — Up to 30x More Work Per Watt: Vera Rubin NVL72 Sets a New Efficiency Standard for AI Agents](https://blogs.nvidia.com/blog/vera-rubin-nvl72-efficiency-ai-agents/) (Shruti Koparkar, August 24, 2026)
- [NVIDIA Technical Blog — Vera Rubin and Blackwell Set a New Standard for Agentic AI Performance per Watt](https://developer.nvidia.com/blog/nvidia-vera-rubin-and-blackwell-set-a-new-standard-for-agentic-ai-performance-per-watt/) (Jamie Li and Eduardo Alvarez, August 24, 2026)
- [SemiAnalysis — AgentX and InferenceX v3](https://newsletter.semianalysis.com/p/agentx-inferencexv3-does-cuda-moat)
- [OpenRouter — DeepSeek V4 adoption and token consumption data](https://openrouter.ai/blog/insights/deepseek-v4-adoption/)

## Frequently Asked Questions

### What did NVIDIA claim about Vera Rubin NVL72?

In posts published on August 24, 2026, NVIDIA reported that Vera Rubin NVL72 delivers up to 30x higher throughput per megawatt and up to 35x lower cost per million tokens than GB300 NVL72 on agentic workloads. The 30x figure was measured on the DeepSeek V4 Pro workload at one specific operating point: 160 output tokens per second per user.

### What is the SemiAnalysis AgentX benchmark?

AgentX is the agentic-coding benchmark in InferenceX, SemiAnalysis's open-source benchmark suite. It replays prerecorded Claude Code sessions turn by turn, preserving each session's context growth, input and output sequence lengths, reasoning time and tool-call latency, so that measured differences reflect the serving stack rather than benchmark-specific tuning.

### Have these results been independently verified?

Not yet. NVIDIA describes them as early results measured by NVIDIA and currently pending SemiAnalysis review. Until that review is published they are vendor-reported numbers.

### Which models were the Vera Rubin results measured on?

Only DeepSeek V4 Pro. NVIDIA separately lists Kimi K3, MiniMax M3, GLM5.3, Qwen3.5 and DeepSeek V4 Pro as models where the Blackwell platform leads on AgentX, but the published Vera Rubin throughput-per-megawatt result is the DeepSeek V4 Pro figure.

### Why does NVIDIA measure throughput per megawatt instead of per GPU?

Because large AI deployments are increasingly limited by available power rather than rack space. NVIDIA's framing is that for power-constrained AI factories, throughput per megawatt determines revenue and cost per million tokens determines the profit margin on that revenue.

### Do the published numbers include the Vera CPU?

No. NVIDIA states the results do not yet reflect Vera CPU performance for tool calling. In the full Vera Rubin platform the Vera CPUs handle tool execution and KV-cache offload, so a substantial part of an agent session's work sits outside what was measured.

---

Source: https://howaiworks.ai/blog/nvidia-vera-rubin-nvl72-agentic-efficiency — HowAIWorks.ai