Diffusion Language Models (DLMs)

Definition

Diffusion language models write text by starting from a block of [MASK] tokens and revealing several of them at once over a few dozen passes, instead of appending one token per pass from left to right. Whether that is faster than the autoregressive models you already use has an honest answer that is not simply yes: committing k tokens per pass cuts the number of sequential passes to 1/k, so the first extra token per pass removes half of them and the eighth removes under 2% more — while every token committed alongside another is sampled without being able to see it.

The thing a diffusion language model can do that no autoregressive model can is look right. In a causal transformer, position i never attends to position i+1; the attention mask forbids it, and a token, once written, is never reconsidered. A masked diffusion model is trained to recover blanked-out positions from everything else in the sequence, so a token still being decided is conditioned on committed tokens on both sides of it. That is where native infilling comes from, and it is why these models are far less lopsided on tasks that run backwards.

Diffusion arrived here from image generation — the general method is a diffusion model — but the mechanism had to change on the way. You can add a fraction of Gaussian noise to a pixel and subtract it again; you cannot add 30% noise to the token cat. Discrete diffusion replaces the continuous noise with an absorbing state — the mask token — so corruption means "blank this position" and denoising means "un-blank it". Everything specific to language models follows from that substitution.

What breaks if you take the speed claim at face value: you budget for the published throughput, deploy on prose or multi-step reasoning rather than boilerplate, and the confidence-based scheduler quietly drops to one or two tokens per step. You are now running a model that recomputes the whole sequence every pass, with no exact KV cache, at roughly the latency of the autoregressive model you replaced and several times its arithmetic.

How It Works

Training: fill in any blank, not the next one

An autoregressive model factorises a sequence exactly one way, p(x) = Π p(x_i | x_<i), and the causal attention mask enforces that single ordering throughout training. Masked diffusion trains a different object. Sample a mask rate t uniformly from 0 to 1, blank each position independently with probability t, and ask the model to recover the blanked positions from the surviving ones. Loss is counted only where the mask is.

The arithmetic of that objective explains most of what follows. With t uniform on [0, 1] the expected fraction of masked positions is 0.5: an autoregressive model gets a gradient at all L positions of a training sequence, a masked diffusion model at about L/2. In exchange it is learning a far larger family of conditionals — any subset of positions given any other subset, rather than one prefix ordering — which is both why it needs more supervision per token and why it can be asked to fill a gap in the middle without any special training format.

That trade lands roughly where you would expect. LLaDA-8B, pre-trained from scratch on 2.3 trillion tokens against Llama 3 8B's 15 trillion, scores 65.9 on MMLU to Llama 3's 65.4 — and 49.7 on BBH to Llama 3's 62.1 (Nie et al., February 2025). One number supports "competitive with Llama 3 8B" and the other does not, which is why the pair is worth more than either alone.

Converting an existing autoregressive checkpoint instead of training from scratch — swap the causal mask for a bidirectional one and continue pretraining — is now the cheaper route to a large diffusion model. That is Simple Continual Pretraining's territory, and the RND1 release is the worked case.

Generation: the unmasking loop

Decoding starts with a canvas of L masked positions. Each forward pass produces a probability distribution for every masked position simultaneously. A schedule then picks k of them to commit — in practice the k the model is most confident about, or every position whose confidence clears a threshold. The rest are returned to [MASK] and predicted again on the next pass, now conditioned on what was just committed.

The asymmetry hiding in that loop is the whole subject. A token committed at pass 3 is visible to a token predicted at pass 4. Two tokens committed in the same pass never see each other at all.

What parallel unmasking saves, and why it saves less each time

Take LLaDA's own efficiency setup: a 256-token response decoded in 256, 128, 64 or 32 sampling steps, which is k = 1, 2, 4 and 8 tokens per pass. Sequential passes are N/k, so the saving against one-token-at-a-time decoding is 1 − 1/k:

The first doubling of k buys half your latency. The fourth buys six percent. Meanwhile the number of tokens sampled blind to one another doubles at every rung. The return falls as 1/k while the risk grows with k — which is why no one runs these models at k = 64, and why "how many tokens can you unmask at once" is the only tuning question that matters.

Why the wall clock falls slower than the step count

Removing passes is not the same as removing time, for two reasons.

First, a diffusion pass costs more than an autoregressive decode step. Bidirectional attention means every position's keys and values depend on the whole partially-masked sequence, and that sequence changes on every pass — so there is no exact KV cache to reuse. An autoregressive decoder reads its cache and computes one new column of attention; a diffusion pass recomputes all L. Producing L tokens one per pass is O(L³) of attention work against the autoregressive decoder's O(L²).

Second, the extra arithmetic is close to free only while the GPU is idle. Decoding is bandwidth- bound — the memory wall page owns that argument, so take it as given here. Having read every weight out of memory, doing 256 positions' worth of multiplication instead of one costs almost nothing, which is exactly why the technique works at all; Google describes DiffusionGemma as moving the bottleneck from memory to raw compute. Once batching has already spent that idle capacity, the free lunch ends.

The other way out of sequential decoding

Speculative decoding spends the same idle arithmetic on the same goal and makes the opposite trade. A small draft model guesses ahead, the large model verifies the guesses in one pass, and the tokens that come out are drawn from precisely the distribution the large model would have produced alone — the only thing given up is compute. Diffusion buys its parallelism in the architecture instead and pays for it in the output. If you want more tokens per step without changing what the model would have said, speculation is the technique; a diffusion language model is a different model with a different distribution.

Block diffusion is what actually ships

Pure diffusion over a thousand-token canvas is rarely what a production system runs. Mercury and DiffusionGemma both denoise in blocks — DiffusionGemma in blocks of 256 tokens — decoding block after block from left to right while unmasking in parallel within a block. That restores an exact KV cache for everything before the current block, bounds how far one bad parallel commit can propagate, and makes the model semi-autoregressive by construction. LLaDA supports the same scheme, which it calls semi-autoregressive remasking, with no retraining. The interesting question about diffusion language models is therefore not whether they replace autoregression, but how large a block can safely be committed at once — and today the answer is a handful of tokens.

Real-World Applications

Low-latency code completion is the deployment that works. Inception Labs' Mercury Coder Mini was measured by Artificial Analysis at 1,109 output tokens per second on an NVIDIA H100 (Inception Labs, June 2025), roughly an order of magnitude above typical autoregressive chat models on comparable hardware. Code fits the mechanism for a reason that generalises: a large share of a completion is indentation, closing brackets, repeated identifiers and boilerplate whose values are pinned by the surrounding context and barely constrain each other. Those are precisely the tokens it is safe to commit together.

Google's own benchmark table shows where the same trade goes wrong. Gemini Diffusion, DeepMind's experimental text-diffusion model, scores 30.9% on LiveCodeBench v6 against Gemini 2.0 Flash-Lite's 28.5% — and 40.4% against 56.5% on GPQA Diamond (DeepMind model page, figures as published). Same pair of models, opposite verdict. The gap tracks how much a token's meaning depends on other tokens being decided at the same moment: tightly-constrained code, fine; multi-step scientific reasoning, not fine.

Editing and infilling need no special format. Filling a gap in the middle of a document, or a function body between an existing signature and an existing return, is the masked diffusion training objective. Autoregressive models reach the same capability only by reordering training data into a fill-in-the-middle format and hoping the reordering generalises. This is the one capability where diffusion is not trading anything away.

The reversal-curse measurement is the cleanest evidence for bidirectional context. On LLaDA's poem-completion task, GPT-4o (the 2024-08-06 checkpoint) scores 82.7 forward and 34.3 reversed; LLaDA-8B Instruct scores 51.8 forward and 45.6 reversed. GPT-4o is far better in the ordinary direction and clearly worse in the other. A 48-point collapse against a 6-point one is the sharpest published number on what causal masking costs — and note that the diffusion model wins by being symmetric, not by being better.

Open weights exist at a serious scale. Google released DiffusionGemma under Apache 2.0 in June 2026 — 26B total parameters with 3.8B active, using the same mixture-of-experts trick as its autoregressive siblings — so evaluating a diffusion language model on your own workload no longer requires a vendor API. LLaDA and Dream remain the research checkpoints most papers benchmark against.

Key Concepts

• Absorbing-state (masked) diffusion vs uniform diffusion: masking is an absorbing state, so a revealed position can simply be frozen and the model's job reduces to "fill the blanks I can see". Uniform-transition diffusion instead swaps a token for any other vocabulary token, which lets the model revise a committed choice but forces it to also learn which tokens are corruptions — a much weaker signal. Every deployed system named on this page is masked diffusion.
• Remasking strategy: how the scheduler chooses which k positions to keep this pass. Drawing them at random is what the sampler's derivation actually calls for; every practical system picks the most confident ones instead, a deliberate deviation from the maths in exchange for coherent output.
• Effective parallelism is an outcome, not a setting: under a confidence threshold, k is whatever the text allows — high across boilerplate, falling to 1 at a genuinely uncertain position. This is why measured throughput on your traffic can look nothing like the number in the announcement.
• Block size (B): the canvas is cut into chunks decoded left to right, so a sequence still has L/B sequential boundaries no matter how parallel the decoding inside one chunk is. This is the dial that trades recoverable KV cache and error containment against raw parallelism.

Challenges

Tokens committed in the same pass are sampled from their marginals, not from the joint. Suppose the model is genuinely undecided between "New York" and "San Francisco", 50/50. Marginally, position one is half New and half San; position two is half York and half Francisco. Commit both in a single pass and the four combinations come out equally likely: half the time you get "New Francisco" or "San York". Nothing malfunctioned — each token was drawn correctly from its own distribution. The correlation lived in the joint, and the joint was never consulted.

That failure compounds with the number of positions, and one case has been worked out exactly. For the task of shuffling a list of n distinct items, ParallelBench derives the accuracy of an ideal model — one whose per-position predictions are exactly right — as a function of k: at k = 1 it tends to 1, and at k = 2 it is (n−1)!! / n!!, which for a ten-item list is 945 / 3840 ≈ 25% and falls to zero as the list grows (Kang et al., ICLR 2026). The ceiling is imposed by the decoding rule, not by the model: two tokens per pass, the smallest parallelism there is, already caps a task that the same model solves every time at k = 1.

Most published quality scores were measured at one token per pass. LLaDA's headline results use pure diffusion sampling with the number of sampling steps set equal to the generation length — exactly one token committed per forward pass. In that configuration the model has no speed advantage at all: it runs L full bidirectional passes to produce L tokens where an autoregressive model runs L cached single-column steps. Benchmark scores and throughput claims for diffusion language models routinely come from different runs, and a table that takes both from the same page is describing two configurations that were never executed together.

Confidence is the wrong signal for the decision it is being asked to make. Threshold unmasking commits every position above γ, which implicitly assumes per-position confidence predicts independence between positions. It does not — a model can be entirely confident about York and about Francisco in precisely the case where committing both is wrong. ParallelBench measures an oracle that picks the best threshold per sample and finds a wide margin over every published strategy, which locates the defect in the decision rule rather than in the models.

Output length is a hyperparameter chosen before the content exists. The canvas has a fixed number of positions, so a diffusion model is told how long its answer will be and pads the remainder with end-of-sequence tokens; autoregressive models simply stop. That means paying for empty positions, and it interacts badly with confidence-based unmasking — LLaDA's authors had to force the confidence of the EOS token to zero on several benchmarks to stop the model terminating early.

The serving stack is built for the other shape. Continuous batching, paged KV cache and prefix caching all assume a monotonically growing cache of committed tokens. Pure diffusion invalidates that cache on every pass, and the approximate caches that recover most of it (Ma et al., 2025) are research code rather than defaults. The throughput a diffusion model reaches on a vendor's tuned stack is not the throughput it reaches on yours.

Future Trends

Adaptive parallelism is the open problem, and everything above points at it. The right k is a property of the next few positions, not of a config file, and no shipped scheduler infers it well — the survey that measured parallelism and generation order across 58 benchmarks in January 2026 concluded that masked diffusion models still trail comparably sized autoregressive ones precisely because parallel decoding weakens inter-token dependencies, and proposed a generate-then-edit paradigm to recover them (Zhong et al., 2026). Expect the next round of gains to come from decoding rules rather than from bigger diffusion models.

The two ways of escaping sequential decoding are converging into one architecture. NVIDIA's TiDAR drafts tokens with a diffusion pass and samples the final output autoregressively inside a single forward pass, using structured attention masks — which is speculative decoding with a diffusion drafter, and reports 4.7x-5.9x the tokens per second of an autoregressive baseline at 1.5B and 8B scale (Chen et al., 2025). Block diffusion was already a hybrid; this is the same convergence run to its conclusion.

Reasoning is the workload the economics favour. A model that spends thousands of test-time compute tokens on a chain of thought nobody reads is paying for sequential latency on text whose only requirement is internal consistency. That is where cutting sequential passes by 4x is worth the most, and it is where the diffusion vendors have aimed their 2026 releases — Inception's Mercury 2, announced in February 2026, is a reasoning model on a diffusion backbone.

Conversion, not training from scratch, is how large diffusion models will keep appearing — and the underrated consequence is coupling. Because a converted model inherits an autoregressive checkpoint, every improvement in autoregressive pretraining becomes an improvement in diffusion pretraining a few weeks later, which makes this a decoding strategy you can convert into rather than a parallel research programme anyone has to fund separately.