---
source: 'https://howaiworks.ai/glossary/diffusion-language-models'
section: glossary
title: Diffusion Language Models (DLMs)
description: >-
  Text models that unmask many tokens per pass instead of writing one at a time
  — what that actually buys in speed, and what it costs in quality.
tags:
  - Diffusion Models
  - LLM
  - Text Generation
  - Inference
  - Deep Learning
  - Efficiency
category: Deep Learning
datePublished: '2025-10-12'
lastUpdated: '2026-07-24'
---

# Diffusion Language Models (DLMs)

> Text models that unmask many tokens per pass instead of writing one at a time — what that actually buys in speed, and what it costs in quality.

## Definition

**Diffusion language models write text by starting from a block of `[MASK]` tokens and revealing
several of them at once over a few dozen passes, instead of appending one token per pass from left
to right.** Whether that is faster than the [autoregressive models](https://howaiworks.ai/glossary/large-language-model)
you already use has an honest answer that is not simply yes: committing *k* tokens per pass cuts
the number of sequential passes to 1/k, so the first extra token per pass removes half of them and
the eighth removes under 2% more — while every token committed alongside another is sampled without
being able to see it.

The thing a diffusion language model can do that no autoregressive model can is look right. In a
causal [transformer](https://howaiworks.ai/glossary/transformer), position *i* never attends to position *i+1*; the
attention mask forbids it, and a token, once written, is never reconsidered. A masked diffusion
model is trained to recover blanked-out positions from *everything else in the sequence*, so a
token still being decided is conditioned on committed tokens on both sides of it. That is where
native infilling comes from, and it is why these models are far less lopsided on tasks that run
backwards.

Diffusion arrived here from [image generation](https://howaiworks.ai/glossary/image-generation) — the general method is a [diffusion model](https://howaiworks.ai/glossary/diffusion-model) — but the mechanism had
to change on the way. You can add a fraction of Gaussian noise to a pixel and subtract it again;
you cannot add 30% noise to the [token](https://howaiworks.ai/glossary/token) `cat`. Discrete diffusion replaces the
continuous noise with an *absorbing state* — the mask token — so corruption means "blank this
position" and denoising means "un-blank it". Everything specific to language models follows from
that substitution.

**What breaks if you take the speed claim at face value:** you budget for the published throughput,
deploy on prose or multi-step reasoning rather than boilerplate, and the confidence-based scheduler
quietly drops to one or two tokens per step. You are now running a model that recomputes the whole
sequence every pass, with no exact [KV cache](https://howaiworks.ai/glossary/kv-cache), at roughly the latency of the
autoregressive model you replaced and several times its arithmetic.

## How It Works

### Training: fill in any blank, not the next one

An autoregressive model factorises a sequence exactly one way, `p(x) = Π p(x_i | x_<i)`, and the
causal attention mask enforces that single ordering throughout training. Masked diffusion trains a
different object. Sample a mask rate *t* uniformly from 0 to 1, blank each position independently
with probability *t*, and ask the model to recover the blanked positions from the surviving ones.
Loss is counted only where the mask is.

The arithmetic of that objective explains most of what follows. With *t* uniform on [0, 1] the
expected fraction of masked positions is **0.5**: an autoregressive model gets a gradient at all
*L* positions of a training sequence, a masked diffusion model at about *L*/2. In exchange it is
learning a far larger family of conditionals — any subset of positions given any other subset,
rather than one prefix ordering — which is both why it needs more supervision per token and why it
can be asked to fill a gap in the middle without any special training format.

That trade lands roughly where you would expect. LLaDA-8B, pre-trained from scratch on 2.3 trillion
tokens against Llama 3 8B's 15 trillion, scores **65.9 on MMLU to Llama 3's 65.4** — and **49.7 on
BBH to Llama 3's 62.1** ([Nie et al., February 2025](https://arxiv.org/abs/2502.09992)). One number
supports "competitive with Llama 3 8B" and the other does not, which is why the pair is worth more
than either alone.

Converting an existing autoregressive checkpoint instead of training from scratch — swap the causal
mask for a bidirectional one and continue pretraining — is now the cheaper route to a large
diffusion model. That is
[Simple Continual Pretraining](https://howaiworks.ai/glossary/simple-continual-pretraining)'s territory, and the
[RND1 release](https://howaiworks.ai/blog/radical-numerics-rnd1-diffusion-model) is the worked case.

### Generation: the unmasking loop

Decoding starts with a canvas of *L* masked positions. Each forward pass produces a probability
distribution for **every** masked position simultaneously. A schedule then picks *k* of them to
commit — in practice the *k* the model is most confident about, or every position whose confidence
clears a threshold. The rest are returned to `[MASK]` and predicted again on the next pass, now
conditioned on what was just committed.

The asymmetry hiding in that loop is the whole subject. A token committed at pass 3 *is* visible to
a token predicted at pass 4. Two tokens committed in the *same* pass never see each other at all.

### What parallel unmasking saves, and why it saves less each time

Take LLaDA's own efficiency setup: a 256-token response decoded in 256, 128, 64 or 32 sampling
steps, which is *k* = 1, 2, 4 and 8 tokens per pass. Sequential passes are *N*/*k*, so the saving
against one-token-at-a-time decoding is `1 − 1/k`:

| Tokens per pass (*k*) | Sequential passes | Passes removed by this doubling |
|---|---|---|
| 1 | 256 | — |
| 2 | 128 | 128 (half the original) |
| 4 | 64 | 64 (a quarter) |
| 8 | 32 | 32 (an eighth) |
| 16 | 16 | 16 (6% of the original) |

The first doubling of *k* buys half your latency. The fourth buys six percent. Meanwhile the number
of tokens sampled blind to one another doubles at every rung. **The return falls as 1/k while the
risk grows with k** — which is why no one runs these models at *k* = 64, and why "how many tokens
can you unmask at once" is the only tuning question that matters.

### Why the wall clock falls slower than the step count

Removing passes is not the same as removing time, for two reasons.

First, a diffusion pass costs more than an autoregressive decode step. Bidirectional attention
means every position's keys and values depend on the whole partially-masked sequence, and that
sequence changes on every pass — so there is no exact [KV cache](https://howaiworks.ai/glossary/kv-cache) to reuse. An
autoregressive decoder reads its cache and computes one new column of attention; a diffusion pass
recomputes all *L*. Producing *L* tokens one per pass is **O(L³)** of attention work against the
autoregressive decoder's **O(L²)**.

Second, the extra arithmetic is close to free only while the GPU is idle. Decoding is bandwidth-
bound — the [memory wall](https://howaiworks.ai/glossary/memory-wall) page owns that argument, so take it as given here.
Having read every weight out of memory, doing 256 positions' worth of multiplication instead of one
costs almost nothing, which is exactly why the technique works at all; Google describes
DiffusionGemma as moving the bottleneck from memory to raw compute. Once
[batching](https://howaiworks.ai/glossary/inference-optimization) has already spent that idle capacity, the free lunch
ends.

### The other way out of sequential decoding

[Speculative decoding](https://howaiworks.ai/glossary/speculative-decoding) spends the same idle arithmetic on the same
goal and makes the opposite trade. A small draft model guesses ahead, the large model verifies the
guesses in one pass, and the tokens that come out are drawn from precisely the distribution the
large model would have produced alone — the only thing given up is compute. Diffusion buys its
parallelism in the architecture instead and pays for it in the output. If you want more tokens per
step *without changing what the model would have said*, speculation is the technique; a diffusion
language model is a different model with a different distribution.

### Block diffusion is what actually ships

Pure diffusion over a thousand-token canvas is rarely what a production system runs. Mercury and
DiffusionGemma both denoise in blocks — DiffusionGemma in blocks of 256 tokens — decoding block
after block from left to right while unmasking in parallel *within* a block. That restores an exact
KV cache for everything before the current block, bounds how far one bad parallel commit can
propagate, and makes the model semi-autoregressive by construction. LLaDA supports the same scheme,
which it calls semi-autoregressive remasking, with no retraining. The interesting question about
diffusion language models is therefore not whether they replace autoregression, but how large a
block can safely be committed at once — and today the answer is a handful of tokens.

## Real-World Applications

**Low-latency code completion is the deployment that works.** Inception Labs' Mercury Coder Mini
was measured by Artificial Analysis at **1,109 output tokens per second** on an NVIDIA H100
([Inception Labs, June 2025](https://arxiv.org/abs/2506.17298)), roughly an order of magnitude
above typical autoregressive chat models on comparable hardware. Code fits the mechanism for a
reason that generalises: a large share of a completion is indentation, closing brackets, repeated
identifiers and boilerplate whose values are pinned by the surrounding context and barely constrain
each other. Those are precisely the tokens it is safe to commit together.

**Google's own benchmark table shows where the same trade goes wrong.** Gemini Diffusion, DeepMind's
experimental text-diffusion model, scores **30.9% on LiveCodeBench v6 against Gemini 2.0
Flash-Lite's 28.5%** — and **40.4% against 56.5% on GPQA Diamond**
([DeepMind model page](https://deepmind.google/models/gemini-diffusion/), figures as published).
Same pair of models, opposite verdict. The gap tracks how much a token's meaning depends on other
tokens being decided at the same moment: tightly-constrained code, fine; multi-step scientific
reasoning, not fine.

**Editing and infilling need no special format.** Filling a gap in the middle of a document, or a
function body between an existing signature and an existing return, *is* the masked diffusion
training objective. Autoregressive models reach the same capability only by reordering training
data into a fill-in-the-middle format and hoping the reordering generalises. This is the one
capability where diffusion is not trading anything away.

**The reversal-curse measurement is the cleanest evidence for bidirectional context.** On LLaDA's
poem-completion task, GPT-4o (the 2024-08-06 checkpoint) scores **82.7 forward and 34.3 reversed**;
LLaDA-8B Instruct scores
**51.8 forward and 45.6 reversed**. GPT-4o is far better in the ordinary direction and clearly
worse in the other. A 48-point collapse against a 6-point one is the sharpest published number on
what causal masking costs — and note that the diffusion model wins by being *symmetric*, not by
being better.

**Open weights exist at a serious scale.** Google released DiffusionGemma under Apache 2.0 in June
2026 — 26B total parameters with 3.8B active, using the same
[mixture-of-experts](https://howaiworks.ai/glossary/mixture-of-experts) trick as its autoregressive siblings — so
evaluating a diffusion language model on your own workload no longer requires a vendor API. LLaDA
and Dream remain the research checkpoints most papers benchmark against.

## Key Concepts

- **Absorbing-state (masked) diffusion vs uniform diffusion**: masking is an absorbing state, so a
  revealed position can simply be frozen and the model's job reduces to "fill the blanks I can
  see". Uniform-transition diffusion instead swaps a token for any other vocabulary token, which
  lets the model revise a committed choice but forces it to also learn *which* tokens are
  corruptions — a much weaker signal. Every deployed system named on this page is masked diffusion.
- **Remasking strategy**: how the scheduler chooses which *k* positions to keep this pass. Drawing
  them at random is what the sampler's derivation actually calls for; every practical system picks
  the most confident ones instead, a deliberate deviation from the maths in exchange for coherent
  output.
- **Effective parallelism is an outcome, not a setting**: under a confidence threshold, *k* is
  whatever the text allows — high across boilerplate, falling to 1 at a genuinely uncertain
  position. This is why measured throughput on your traffic can look nothing like the number in
  the announcement.
- **Block size (*B*)**: the canvas is cut into chunks decoded left to right, so a sequence still
  has *L*/*B* sequential boundaries no matter how parallel the decoding inside one chunk is. This
  is the dial that trades recoverable KV cache and error containment against raw parallelism.

## Challenges

**Tokens committed in the same pass are sampled from their marginals, not from the joint.** Suppose
the model is genuinely undecided between "New York" and "San Francisco", 50/50. Marginally,
position one is half `New` and half `San`; position two is half `York` and half `Francisco`. Commit
both in a single pass and the four combinations come out equally likely: **half the time you get
"New Francisco" or "San York"**. Nothing malfunctioned — each token was drawn correctly from its
own distribution. The correlation lived in the joint, and the joint was never consulted.

That failure compounds with the number of positions, and one case has been worked out exactly. For
the task of shuffling a list of *n* distinct items, ParallelBench derives the accuracy of an
*ideal* model — one whose per-position predictions are exactly right — as a function of *k*: at
*k* = 1 it tends to 1, and at *k* = 2 it is `(n−1)!! / n!!`, which for a ten-item list is
`945 / 3840 ≈ 25%` and falls to zero as the list grows
([Kang et al., ICLR 2026](https://arxiv.org/abs/2510.04767)). The ceiling is imposed by the
decoding rule, not by the model: two tokens per pass, the smallest parallelism there is, already
caps a task that the same model solves every time at *k* = 1.

**Most published quality scores were measured at one token per pass.** LLaDA's headline results use
pure diffusion sampling with the number of sampling steps set equal to the generation length —
exactly one token committed per forward pass. In that configuration the model has no speed advantage
at all: it runs *L* full bidirectional passes to produce *L* tokens where an autoregressive model
runs *L* cached single-column steps. Benchmark scores and throughput claims for diffusion language
models routinely come from different runs, and a table that takes both from the same page is
describing two configurations that were never executed together.

**Confidence is the wrong signal for the decision it is being asked to make.** Threshold unmasking
commits every position above γ, which implicitly assumes per-position confidence predicts
*independence between* positions. It does not — a model can be entirely confident about `York` and
about `Francisco` in precisely the case where committing both is wrong. ParallelBench measures an
oracle that picks the best threshold per sample and finds a wide margin over every published
strategy, which locates the defect in the decision rule rather than in the models.

**Output length is a hyperparameter chosen before the content exists.** The canvas has a fixed
number of positions, so a diffusion model is told how long its answer will be and pads the
remainder with end-of-sequence tokens; autoregressive models simply stop. That means paying for
empty positions, and it interacts badly with confidence-based unmasking — LLaDA's authors had to
force the confidence of the EOS token to zero on several benchmarks to stop the model terminating
early.

**The serving stack is built for the other shape.** Continuous batching, paged KV cache and prefix
caching all assume a monotonically growing cache of committed tokens. Pure diffusion invalidates
that cache on every pass, and the approximate caches that recover most of it
([Ma et al., 2025](https://arxiv.org/abs/2505.15781)) are research code rather than defaults. The
throughput a diffusion model reaches on a vendor's tuned stack is not the throughput it reaches on
yours.

## Future Trends

**Adaptive parallelism is the open problem, and everything above points at it.** The right *k* is a
property of the next few positions, not of a config file, and no shipped scheduler infers it well —
the survey that measured parallelism and generation order across 58 benchmarks in January 2026
concluded that masked diffusion models still trail comparably sized autoregressive ones precisely
because parallel decoding weakens inter-token dependencies, and proposed a generate-then-edit
paradigm to recover them ([Zhong et al., 2026](https://arxiv.org/abs/2601.15593)). Expect the next
round of gains to come from decoding rules rather than from bigger diffusion models.

**The two ways of escaping sequential decoding are converging into one architecture.** NVIDIA's
TiDAR drafts tokens with a diffusion pass and samples the final output autoregressively inside a
single forward pass, using structured attention masks — which is speculative decoding with a
diffusion drafter, and reports 4.7x-5.9x the tokens per second of an autoregressive baseline at
1.5B and 8B scale ([Chen et al., 2025](https://arxiv.org/abs/2511.08923)). Block diffusion was
already a hybrid; this is the same convergence run to its conclusion.

**Reasoning is the workload the economics favour.** A model that spends thousands of
[test-time compute](https://howaiworks.ai/glossary/test-time-compute) tokens on a chain of thought nobody reads is
paying for sequential latency on text whose only requirement is internal consistency. That is where
cutting sequential passes by 4x is worth the most, and it is where the diffusion vendors have
aimed their 2026 releases — Inception's Mercury 2, announced in February 2026, is a reasoning model
on a diffusion backbone.

**Conversion, not training from scratch, is how large diffusion models will keep appearing** — and
the underrated consequence is coupling. Because a converted model inherits an autoregressive
checkpoint, every improvement in autoregressive pretraining becomes an improvement in diffusion
pretraining a few weeks later, which makes this a decoding strategy you can convert into rather
than a parallel research programme anyone has to fund separately.

## Frequently Asked Questions

### What is a diffusion language model?

A model that writes text by starting from a block of mask tokens and revealing several of them per forward pass, instead of appending one token per pass from left to right. Every position can see every other position while it is being decided, which is the thing autoregressive models cannot do.

### Are diffusion language models actually faster than autoregressive ones?

They can be, and the throughput numbers are real — but the speedup is not free and not automatic. Committing k tokens per pass cuts the number of sequential passes to 1/k, so the first extra token per pass removes half the steps and the eighth removes barely 1%, while every token committed alongside another is sampled without seeing it. On text with strong token-to-token dependencies, k collapses back toward 1 and the model does far more arithmetic per token than an autoregressive model would.

### Why does parallel decoding hurt quality?

Tokens revealed in the same step are each sampled from their own marginal distribution, not from the joint. If a model is genuinely 50/50 between 'New York' and 'San Francisco' and commits both positions at once, a quarter of the time it writes 'New Francisco' and a quarter of the time 'San York'. Each token was sampled correctly; the correlation between them was never consulted.

### How is this different from speculative decoding?

Speculative decoding gets more tokens per step from a small draft model plus a verification pass, and the output is provably identical in distribution to what the large model would have produced on its own. A diffusion language model is a different model with a different distribution — it buys parallelism in the architecture and pays for it in output quality rather than in a verification pass.

### Which diffusion language models can I actually use?

Inception Labs' Mercury is the commercial one, aimed at low-latency code completion. Google released DiffusionGemma under Apache 2.0 in June 2026 with 26B total and 3.8B active parameters, and Gemini Diffusion remains an experimental demo. LLaDA and Dream are the open research models most papers benchmark against.

## Related

### Related terms

- [Speculative Decoding](https://howaiworks.ai/glossary/speculative-decoding)
- [Large Language Model (LLM)](https://howaiworks.ai/glossary/large-language-model)
- [Transformer](https://howaiworks.ai/glossary/transformer)
- [KV Cache](https://howaiworks.ai/glossary/kv-cache)
- [Text Generation](https://howaiworks.ai/glossary/text-generation)
- [Simple Continual Pretraining (SCP)](https://howaiworks.ai/glossary/simple-continual-pretraining)

---

Source: https://howaiworks.ai/glossary/diffusion-language-models — HowAIWorks.ai