---
source: 'https://howaiworks.ai/blog/nvidia-blackwell-moe-inference'
section: blog
title: 'NVIDIA Blackwell: Performance Leaps for MoE'
description: >-
  Discover how NVIDIA Blackwell and TensorRT-LLM deliver up to 2.8x throughput
  increases for Mixture of Experts (MoE) models like DeepSeek-R1.
date: '2026-01-21'
author: HowAIWorks Team
tags:
  - NVIDIA
  - Blackwell
  - MoE
  - Inference
  - TensorRT-LLM
  - DeepSeek-R1
  - AI Infrastructure
readingTime: 4 minutes
---

# NVIDIA Blackwell: Performance Leaps for MoE

> Discover how NVIDIA Blackwell and TensorRT-LLM deliver up to 2.8x throughput increases for Mixture of Experts (MoE) models like DeepSeek-R1.

## Introduction

As AI models grow in complexity and intelligence, the demand for efficient, high-performance inference platforms has never been higher. NVIDIA's Blackwell architecture, combined with the latest software optimizations in TensorRT-LLM, is setting new benchmarks for Mixture of Experts (MoE) models.

Recent benchmarks show that NVIDIA Blackwell is delivering massive performance leaps, particularly for sparse MoE architectures like DeepSeek-R1. This article explores the architectural and software innovations driving these gains.

Every optimization below is an attack on the same constraint. Generating a token means reading the model's weights out of memory, and the arithmetic done on the way past them is trivial by comparison — so inference is limited by memory bandwidth, not by compute. That is the [memory wall](https://howaiworks.ai/glossary/memory-wall). Sparse MoE reads fewer weights per token; [4-bit formats](https://howaiworks.ai/glossary/quantization) make each weight smaller; and splitting prefill from decode separates the compute-bound phase from the memory-bound one, which is why the [KV cache](https://howaiworks.ai/glossary/kv-cache) can then be managed independently.

## The Power of GB200 NVL72 for MoE

The [NVIDIA GB200 NVL72](https://www.nvidia.com/en-us/data-center/gb200-nvl72/) is a rack-scale platform designed to handle the most demanding AI workloads. It connects 72 Blackwell GPUs using fifth-generation NVLink, offering a staggering 1,800 GB/s of bidirectional bandwidth between all chips.

- **Optimized for Sparse MoE**: Sparse MoE models, like the 671B parameter DeepSeek-R1, activate only a fraction of their parameters per token (37B for DeepSeek-R1). This requires high-speed data exchange between experts.
- **NVFP4 Data Format**: Blackwell introduces hardware acceleration for the NVFP4 format—a 4-bit floating point format that maintains accuracy while significantly boosting compute efficiency.
- **Disaggregated Serving**: This technique splits prefill and decode operations across different GPUs, taking full advantage of the NVL72's architecture to minimize latency.

## Software Innovations in TensorRT-LLM

Hardware alone isn't enough; NVIDIA’s software stack plays a critical role. The [NVIDIA TensorRT-LLM](https://developer.nvidia.com/tensorrt-llm) library has introduced several enhancements that have increased Blackwell's throughput by up to 2.8x in just three months.

- **Programmatic Dependent Launch (PDL)**: PDL reduces kernel launch latencies, which is essential for maintaining high throughput across various interactivity levels.
- **Kernel Optimizations**: Low-level tweaks ensure that Blackwell Tensor Cores are utilized as efficiently as possible.
- **All-to-All Communication**: New primitives eliminate intermediate buffers on the receiver side, streamlining the data flow between experts in a distributed system.

## HGX B200: Performance for Air-Cooled Deployments

For environments that don't require full rack-scale liquid cooling, the [NVIDIA HGX B200](https://www.nvidia.com/en-us/data-center/hgx/) platform provides exceptional performance. It uses eight Blackwell GPUs connected via NVLink and leverages two key technologies:

1.  **Multi-Token Prediction (MTP)**: This significantly increases throughput across all tested input/output sequence lengths.
2.  **NVFP4 Acceleration**: By utilizing the NVFP4 format via the full NVIDIA software stack, HGX B200 achieves a substantial throughput boost without sacrificing model accuracy.

## Conclusion

NVIDIA's commitment to continuous optimization across the entire stack—from hardware architecture to low-level software libraries—is delivering unprecedented value. By leveraging the full capabilities of the Blackwell platform, enterprises can serve more users with lower latency and higher efficiency.

As models like DeepSeek-R1 continue to push the boundaries of what's possible, platforms like GB200 NVL72 and HGX B200 will be the foundation for the next generation of intelligent AI applications.

## Sources

- [Delivering Massive Performance Leaps for MoE Inference on NVIDIA Blackwell](https://developer.nvidia.com/blog/delivering-massive-performance-leaps-for-mixture-of-experts-inference-on-nvidia-blackwell/)
- [NVIDIA GB200 NVL72](https://www.nvidia.com/en-us/data-center/gb200-nvl72/)
- [NVIDIA TensorRT-LLM GitHub](https://github.com/NVIDIA/TensorRT-LLM)

## Frequently Asked Questions

### What is the performance leap for DeepSeek-R1 on Blackwell?

NVIDIA Blackwell GPUs have seen a throughput increase of up to 2.8x for DeepSeek-R1 inference over the past three months due to software optimizations.

### What are the key Blackwell platforms for MoE inference?

The primary platforms are GB200 NVL72 (rack-scale with 72 GPUs) and HGX B200 (eight GPUs connected via NVLink).

### What technologies drive these performance gains?

Key technologies include NVFP4 data format, Programmatic Dependent Launch (PDL), and optimized all-to-all communication primitives.

### How does GB200 NVL72 handle large MoE models?

It uses fifth-generation NVLink to provide 1,800 GB/s of bidirectional bandwidth, enabling frequent data exchanges required by sparse MoE architectures.

---

Source: https://howaiworks.ai/blog/nvidia-blackwell-moe-inference — HowAIWorks.ai