---
source: 'https://howaiworks.ai/glossary/synthetic-data'
section: glossary
title: Synthetic Data
description: >-
  Data generated by a model instead of collected from the world: what synthetic
  data is, where it helps, and why recursive use triggers model collapse.
tags:
  - Machine Learning
  - Generative AI
  - Model Training
  - data quality
  - Privacy
  - Foundation Models
category: Machine Learning
datePublished: '2026-07-24'
lastUpdated: '2026-07-24'
---

# Synthetic Data

> Data generated by a model instead of collected from the world: what synthetic data is, where it helps, and why recursive use triggers model collapse.

## Definition

Synthetic data is data produced by a model or a simulation rather than collected from the
world. A generative model learns an approximation of some real distribution — images, text,
patient records — and then emits fresh samples from that approximation, which are used to train,
test or fine-tune another model.

The distinction that matters most is against [data augmentation](https://howaiworks.ai/glossary/data-augmentation), and
it is easy to blur. Augmentation *transforms an example you already have*: rotate a photo, paraphrase
a sentence, add noise to an audio clip. There is a real datum underneath every augmented one, and
its label comes along for free — a rotated cat is still a cat. Synthetic data has no real datum
underneath it. It is a sample from another model, so it inherits that model's blind spots, and its
"label" is an assertion the generator makes, not an observation anyone verified. That single
difference — transform of a real thing versus fresh draw from a model — is what makes synthetic
data both far more flexible and far more dangerous. The specific danger has a name, *model collapse*,
and it is the reason this term is worth understanding before you reach for it.

## How It Works

Synthetic data is generated three broad ways, and they do not carry the same risk.

The first is **sampling from a generative model** trained on the same kind of data you want more of
— a [GAN](https://howaiworks.ai/glossary/generative-adversarial-network) or [diffusion model](https://howaiworks.ai/glossary/diffusion-model)
for images, a [large language model](https://howaiworks.ai/glossary/large-language-model) for text. The second is
**distillation**: you take a stronger "teacher" model, have it produce inputs and answers, and
[fine-tune](https://howaiworks.ai/glossary/fine-tuning) a smaller "student" on that transcript — the student learns to
imitate the teacher for a fraction of the cost. (This is closely related to
[knowledge distillation](https://howaiworks.ai/glossary/knowledge-distillation), and it is how a great deal of open-model
instruction tuning and supervised fine-tuning is actually done.) The third is **simulation**:
a physics engine or game world renders labelled examples — a self-driving stack sees a rendered
pedestrian, and the simulator already knows the exact bounding box because it drew it.

The first two are model-generated and share a failure mode; simulation is grounded in a hand-built
world rather than in a learned model of your data, so it does not feed on itself the same way.

### The mechanism that breaks: statistical approximation error

The 2024 *Nature* paper that formalised the risk (Shumailov et al., "AI models collapse when trained
on recursively generated data") isolates the primary cause and it is not exotic. It is *finite
sampling*. Any finite sample can, by chance, miss a low-probability event — and the model trained on
that sample can no longer produce what its training data did not contain. Do this repeatedly — train a
model, sample from it, train the next model on those samples — and each round shaves a little more off
the tails of the distribution. The paper's phrase is that the process "first starts with tails
disappearing"; carried far enough it converges to what they prove is a *delta function*, a single
point with zero variance. They call the two stages **early model collapse** (the tails go) and
**late model collapse** (the distribution narrows to near-nothing).

The arithmetic is worth doing once, because it shows the loss is not gradual. Suppose a genuine but
uncommon pattern — a rare disease presentation, an unusual code idiom, a minority dialect — appears in
about 1 in 1,000 real examples, so its probability is `p = 0.001`. A model trained on that data
reproduces it at roughly the same rate. Now generate a synthetic corpus of `n = 1,000` examples to
train the next model. The *expected* number of examples containing the pattern is `n × p = 1` — but
the probability that the corpus contains **none** of it is:

```
(1 − p)ⁿ = (0.999)^1000 ≈ 1/e ≈ 0.37
```

So a single "train, generate, retrain" round has about a 37% chance of dropping that pattern
entirely. And because a model cannot regenerate what it never saw, the loss is one-way: once the
pattern is gone from a generation's data it is gone for good. Chain the rounds and the survival
probability is roughly `0.63` per generation, so after ten it is `0.63¹⁰ ≈ 0.01` — about a 1% chance
the pattern is still there. This is exactly the *Nature* paper's discrete argument, where a state with
probability at or below `1/M` (M being the sample size) is expected to vanish: the tail does not fade,
it falls off a cliff, one rare feature at a time.

Empirically the authors demonstrated it on language. They fine-tuned Meta's OPT-125M model on the
wikitext2 corpus, generated a same-size dataset from it, fine-tuned the next generation on *that*, and
repeated for nine generations. With no real data retained, quality degraded monotonically across the
generations; keeping a slice of the original data each round left only minor degradation. Crucially,
one distillation *pass* is not collapse — collapse is what happens when a model's own output becomes
the training set for its successor, again and again, with the real distribution drained out of the
loop.

## Real-World Applications

Synthetic data is not a research curiosity; it is in production wherever real data is scarce, unsafe
to collect, or illegal to share.

- **Distilling a stronger model.** Stanford's Alpaca (2023) fine-tuned a base LLaMA model on about
  52,000 instruction-response examples generated by an OpenAI model, at a data cost of a few hundred
  dollars. A large share of today's open instruction- and chat-tuned models are trained this way — the
  student cannot exceed the teacher, but it can get most of the way there cheaply. This is the "filling
  in behaviour by copying a stronger model" use, and it works precisely because it is a single pass, not
  a recursive loop.
- **Curated synthetic pretraining.** Microsoft's Phi models ("Textbooks Are All You Need", 2023 onward)
  were trained substantially on synthetically generated textbook-style text and exercises, and a small
  Phi model matched far larger ones on reasoning and code. It was a headline demonstration that
  *curated* synthetic data can beat raw web scrape — when a stronger model and heavy filtering stand
  between the generator and the training set.
- **Rare and dangerous edge cases.** Autonomous-driving programmes (Waymo, Cruise) and robotics teams
  train and test perception on simulation — NVIDIA's Omniverse Replicator and Isaac Sim render labelled
  scenes on demand. A child darting between parked cars is unsafe and rare to capture on the road, but
  trivial to render a million times with pixel-perfect labels, because the simulator knows the ground
  truth it drew.
- **Privacy-preserving records.** In finance and healthcare the real rows cannot leave the building.
  Tools such as MOSTLY AI, Gretel and the open-source Synthetic Data Vault fit a generative model to a
  sensitive table and emit statistically similar fake rows that can be shared for analytics and model
  development — with the caveat, below, that "synthetic" is not automatically "private."

## Key Concepts

**The real-data anchor.** The single most important dial is how much genuine data stays in the loop.
The *Nature* study's mild-degradation setting kept 10% of the original data at every generation; a
growing line of follow-up work finds that *accumulating* real and synthetic data, rather than
*replacing* real with synthetic, arrests the drift. Synthetic data is a supplement, and it behaves
very differently as a replacement.

**Coverage versus fidelity.** A synthetic example can look completely realistic on its own while the
*set* systematically under-represents the tails. This is what makes collapse insidious: the failure is
at the distribution level, not the example level, so eyeballing a handful of convincing samples tells
you nothing about whether the rare cases are still there.

**Label fidelity.** A synthetic label is a claim, not a measurement. The robust pattern — the same one
the [data augmentation](https://howaiworks.ai/glossary/data-augmentation) page recommends for generative augmentation — keeps
a real example or an independent verifier in the loop rather than trusting a class name and a generator.

## Challenges

**Model collapse is the headline risk, and it is specifically the recursive regime.** Training a model
on its own (or other models') output, generation after generation, with the real distribution drained
out, drives the tails to zero — the mechanism worked through above. This is a real and, as of the 2024
finding, well-characterised threat as the open web fills with model-generated text and images and clean
human data becomes harder to isolate. A single distillation pass is not this; recursive self-training is.

**Narrowing shows up on the tail, not the average.** Even one generation quietly trims minority modes.
Standard validation is dominated by the common cases, so an aggregate metric can look healthy while
rare-case performance rots — you notice it on the unusual query, the underrepresented group, the long
tail, which is exactly where a validation set is thinnest. It reads less like a loud failure and more
like a slow, invisible [overfitting](https://howaiworks.ai/glossary/overfitting) to the head of the distribution.

**Errors compound instead of averaging out.** Any systematic mistake or bias in the generator becomes
ground truth for the next model, which reproduces and amplifies it. This is adjacent to
[data poisoning](https://howaiworks.ai/glossary/data-poisoning): there the corruption is adversarial and deliberate, here it
is self-inflicted, but in both a bad signal in the training data propagates forward and is hard to undo.

**Synthetic is not automatically private.** A generative model can memorise and regurgitate the real
records it was fit on, and membership-inference and reconstruction attacks can recover individuals from
"anonymous" synthetic tables. Privacy has to be enforced with a mechanism — differential privacy during
training, for instance — not assumed from the fact that the output is generated. Treating synthetic data
as safe to share by default is a [privacy](https://howaiworks.ai/glossary/privacy) failure waiting to happen.

**Provenance is getting harder.** The *Nature* authors' closing point is economic: as model output
saturates the web, the value of data provably collected from genuine human interaction goes *up*, and
the ability to curate a clean real corpus — or even to tell human from machine text — goes down. That
makes the real-data anchor above not just a training choice but a supply problem that gets worse over
time.

Note: this term was last reviewed in July 2026. The named systems, tools and dated results cited above
(the OPT-125M experiment, the Phi and Alpaca examples, the specific privacy tools) are the parts of this
page most likely to age; the finite-sampling mechanism and its arithmetic do not.

## Frequently Asked Questions

### What is the difference between synthetic data and data augmentation?

Augmentation transforms a real example you already have — cropping an image, paraphrasing a sentence — so the label carries over from the original. Synthetic data is generated wholesale: a fresh sample drawn from a model, with no real example underneath it and a label the model asserts rather than one you observed.

### Does training on synthetic data always cause model collapse?

No. Collapse is the failure of the recursive, replace-everything regime — training a model on its own output, then training the next model on that, with no fresh real data. A single distillation pass from a stronger model, or synthetic data mixed with real data, does not collapse; the 2024 Nature study that named the effect found that keeping even 10% real data each round sharply reduced it.

### Is synthetic data private by default?

No. A generative model can memorise and reproduce real records, so 'fake' rows can still leak the individuals they were fit on. Membership-inference and reconstruction attacks demonstrate this; privacy has to be enforced with a mechanism such as differential privacy, not assumed because the output is synthetic.

### Why do teams use synthetic data if it carries these risks?

It fills gaps real data cannot: rare or dangerous edge cases you cannot safely collect, private records you are not allowed to share, and cheap distillation of a stronger model's behaviour into a smaller one. The risks are manageable when synthetic data supplements real data rather than replacing it.

## Related

### Related terms

- [Data Augmentation](https://howaiworks.ai/glossary/data-augmentation)
- [Knowledge Distillation](https://howaiworks.ai/glossary/knowledge-distillation)
- [Fine-tuning (FT)](https://howaiworks.ai/glossary/fine-tuning)
- [Generative Adversarial Network (GAN)](https://howaiworks.ai/glossary/generative-adversarial-network)
- [Diffusion Model](https://howaiworks.ai/glossary/diffusion-model)
- [Data Poisoning](https://howaiworks.ai/glossary/data-poisoning)

---

Source: https://howaiworks.ai/glossary/synthetic-data — HowAIWorks.ai