---
source: 'https://howaiworks.ai/glossary/foundation-models'
section: glossary
title: Foundation Models
description: >-
  Not a marketing word for a big AI model: a foundation model is trained once on
  broad data and reused as the base for many downstream tasks.
tags:
  - Foundation Models
  - pre-trained models
  - transfer learning
  - AI governance
  - Open Source
  - AI Infrastructure
category: Artificial Intelligence
datePublished: '2025-07-16'
lastUpdated: '2026-07-24'
---

# Foundation Models

> Not a marketing word for a big AI model: a foundation model is trained once on broad data and reused as the base for many downstream tasks.

## Definition

A **foundation model** is a model trained once on broad data and then adapted to many different
downstream tasks. That is a technical category with a birthday, not a compliment: the term was
coined in *"On the Opportunities and Risks of Foundation Models"*, posted to arXiv on
[16 August 2021](https://arxiv.org/abs/2108.07258) by **114 authors** at Stanford's newly formed
Center for Research on Foundation Models, which defined one as "any model that is trained on broad
data (generally using self-supervision at scale) that can be adapted (e.g., fine-tuned) to a wide
range of downstream tasks."

Read that definition again and notice what is missing. There is no parameter count in it, no
modality, no architecture, no benchmark score and no vendor. Nothing about being big. Arc
Institute's [**Evo 2**](https://www.nature.com/articles/s41586-026-10176-5), a DNA model with 40
billion parameters trained on 9.3 trillion nucleotides and published in *Nature*, is a foundation
model in the full sense — other labs adapt it for variant-effect prediction and genome design
rather than training their own. A 500-billion-parameter recommender trained inside
one company to rank one feed is not, however large it is. The criterion is not the model. It is the
number of things standing on it.

That is why a new word was needed in 2021 at all. "[Large language model](https://howaiworks.ai/glossary/large-language-model)"
was too narrow — the same phenomenon was appearing in vision and biology. "[Pre-trained
model](https://howaiworks.ai/glossary/pre-trained-models)" described the training recipe and not the role. The authors
wanted a name for a *dependency structure*: a small number of models that are trained once, at
enormous cost, and then sit underneath thousands of products that nobody retrains.

**What breaks if you read it as a size class.** You lose the argument the term was built to carry.
The report's central claim is about **homogenization** — the consolidation of thousands of
applications onto a handful of bases — and it is deliberately double-edged: "Homogenization
provides powerful leverage but demands caution, as the defects of the foundation model are
inherited by all the adapted models downstream." Elsewhere the report is blunter: "foundation
models are a high-leverage single point of failure." If you are building on one, its bugs are your
bugs, its biases are your biases, and its next version ships on its schedule, not yours.

## How It Works

### The adaptation step is the least distinctive part

Pretrain on broad data with [self-supervision](https://howaiworks.ai/glossary/self-supervised-learning), then adapt by
[fine-tuning](https://howaiworks.ai/glossary/fine-tuning), prompting or [transfer learning](https://howaiworks.ai/glossary/transfer-learning).
That workflow long predates the word "foundation model" — it is what ImageNet-pretrained CNNs did in
2014 — and those pages own the mechanics. What changed in 2020–2021 was not the recipe but the
*ratio*: how few models are pretrained, and how many things are adapted from each one.

### The base layer is small, and it is small for a priced reason

Training a frontier base model is a capital expenditure, and you can put a number on it without
quoting anyone's press release. Meta's model card for Llama 3.1 405B reports **30.84 million
H100-hours** for that single run. On-demand H100 rental sat at roughly **$2–3 per GPU-hour** across
specialist GPU clouds in mid-2026, so the arithmetic on compute alone is:

> 30.84 × 10⁶ GPU-hours × $2/hour ≈ **$62 million**, and about **$93 million** at $3/hour —
> before data acquisition, before salaries, and before the failed runs that never ship.

That price is the whole explanation for the shape of the ecosystem. When a single base run is a
tens-of-millions-of-dollars capital expenditure, only a handful of organisations on Earth can pay
to train one from scratch, and everyone else adapts what those few produce. That is a structural
claim, not a headcount: it follows from the arithmetic above and holds whatever this quarter's
exact census turns out to be. For scale, a June 2025 reading of
[Epoch AI's tracker](https://epoch.ai/data-insights/models-over-1e25-flop) counted **33 publicly
announced models trained at or above 10²⁵ FLOP, from just 12 distinct developers** — a count that
moves as new runs land, but not by an order of magnitude, because the cost floor that produces it
does not. (The [FLOPs](https://howaiworks.ai/glossary/flops) page owns the compute arithmetic itself, and
[scaling laws](https://howaiworks.ai/glossary/scaling-laws) owns the question of what that compute buys.)

Note that the threshold and the category are different things. Run the standard `6ND` estimate over
Evo 2 — 6 × 4.0 × 10¹⁰ parameters × 9.3 × 10¹² tokens ≈ **2.2 × 10²⁴ FLOP** — and you get a model
at roughly a *quarter* of the EU's systemic-risk line that is still, indisputably, a foundation
model. "Frontier model" is a claim about training compute. "Foundation model" is a claim about how
many things depend on you. They are frequently true of the same model and they are not the same
statement.

### The layer above it is enormous

Hugging Face's [spring 2026 review](https://huggingface.co/blog/huggingface/state-of-os-hf-spring-2026)
of its own Hub reports **more than 2 million public models** and
13 million users. Almost none of those 2 million were trained from scratch. Alibaba's Qwen family
alone accounts for **more than 113,000 derivative models** — more, Hugging Face notes, than Google
and Meta's derivative ecosystems combined. And the concentration is sharper on the demand side than
on the supply side: **the top 200 models, 0.01% of everything on the Hub, take 49.6% of all
downloads**, while roughly half of all models have fewer than 200 downloads ever.

Put the two counts next to each other and the structure is an hourglass. A dozen organisations at
the waist; six figures of adapted models and an unknown number of shipped products in the bulb
above. That ratio — not the parameter count — is what the word "foundation" was chosen to describe.

### Homogenization is the feature and the failure mode, and they are one mechanism

Because the same base is underneath everything, an improvement to it improves everything at once.
So does a regression. The clearest public demonstration is OpenAI's GPT-4o update of **25 April
2025**, which tuned on short-term user feedback and produced a [sycophantic](https://howaiworks.ai/glossary/sycophancy) model that agreed with users far too
readily — including when they were wrong or unwell.
[OpenAI rolled it back](https://openai.com/index/sycophancy-in-gpt-4o/) on **29 April**, four days
later.

Look at what those four days cost people who did not cause them. Every product calling that
endpoint — support agents, tutoring apps, journaling tools, therapy-adjacent chatbots — changed
personality simultaneously, without shipping a line of code. None of them could roll back; the fix
was not theirs to deploy. This is exactly the "defects inherited downstream" sentence, running in
production, in an ordinary week, from a change that was not even a bug in the usual sense. The
dependency is invisible in your repository and total in your product.

## Real-World Applications

**Operational weather forecasting.** ECMWF put its Artificial Intelligence Forecasting System
([AIFS](https://www.ecmwf.int/en/about/media-centre/news/2025/ecmwfs-ai-forecasts-become-operational))
into operations on **25 February 2025**, running alongside the physics-based model that had
been the European standard for decades. It beats that model on several measures, including tropical
cyclone tracks by up to **20%**, and produces a forecast for roughly **1,000× less energy**. This is
the best argument that "foundation model" is not a synonym for "chatbot": AIFS generates no content,
answers no prompts, and is a foundation model in exactly the CRFM sense — one broad-data model that
downstream users adapt to their own variables and regions.

**Genomics.** Evo 2 was trained across all domains of life and released with weights, training code
and training data — an unusual level of disclosure. Its 1-megabase context lets it read whole
genes with their regulatory neighbourhoods, and downstream groups adapt it rather than retrain it,
which is the entire economic point: the 9.3-trillion-nucleotide run happens once, and the field
builds on the result. Related work on [protein folding](https://howaiworks.ai/glossary/protein-folding) follows the same
pattern.

**The enterprise application layer.** Menlo Ventures'
[December 2025 enterprise survey](https://menlovc.com/perspective/2025-the-state-of-generative-ai-in-the-enterprise/)
found the LLM API market split **Anthropic 40%, OpenAI 27%, Google 21%** — **88% of enterprise usage across
three suppliers**, in a category that grew from $1.7 billion in 2023 to $37 billion in 2025. Anyone
buying AI in that market is, whether the contract says so or not, taking a dependency on one of
three research organisations' training decisions.

**Procurement and regulation.** Under Article 51 of the EU AI Act, a general-purpose model is
*presumed* to carry systemic risk once cumulative training compute exceeds **10²⁵ FLOP**, and the
obligations that follow have applied since **2 August 2025**. The practical consequence for a
downstream buyer is that a compliance tier is set by a training run someone else finished years
earlier — see [AI governance](https://howaiworks.ai/glossary/ai-governance) for what those duties actually are.

## Key Concepts

- **The base model and the model you call are different artefacts.** What a provider serves is a
  base model plus instruction tuning, preference optimisation, safety classifiers and a system
  prompt. Two products on "the same" foundation model can behave very differently, and a provider
  can change the layers above the base without changing the base at all.
- **Open weights is not open data.** You can download Llama or Qwen and still not know what they
  were trained on. That gap is exactly where inherited copyright, privacy and bias exposure lives,
  and it is the subdomain where transparency scores are consistently worst.
- **Adaptation is cheap only relative to pretraining.** The asymmetry — tens of millions to train,
  hundreds of dollars to fine-tune — is what makes the hourglass stable. It is also why "just train
  your own" is advice that costs more than most companies are worth.
- **Capability you did not ask for arrives anyway.** Because the base is trained on broad data, an
  adapted model retains capabilities irrelevant to your product, including ones you would rather it
  did not have. Narrowing the task does not narrow the model.

## Challenges

**Correlated failure has no diversification story.** In most supply chains you can second-source.
Here, the alternates were trained on overlapping web corpora with similar objectives, so they share
failure modes: the same [hallucination](https://howaiworks.ai/glossary/hallucinations) tendencies, similar prompt-injection
susceptibility, similar blind spots in low-resource languages. Switching providers protects you
against a single company's outage and barely at all against the class of defect the 2021 report
warned about. Real mitigation looks like verification you own — evaluations on your data, guardrails
outside the model, human review where the cost of being wrong is high — not a second vendor.

**You cannot audit what nobody discloses, and disclosure is going backwards.** Stanford's
[**Foundation Model Transparency Index**](https://crfm.stanford.edu/fmti/December-2025/index.html),
in its third edition (December 2025), scored 13 developers
against 100 indicators. The mean was **41 out of 100** — a **17-point fall** from the 2024 edition,
leaving it only 4 points above where the index started in 2023. The spread is extreme: **IBM scored
95, xAI and Midjourney 14**. Among developers scored in both years, **OpenAI fell 14 points, Meta 29
and Mistral 37**. Training data and training compute remain the most opaque areas of all. So
dependence on foundation models is deepening while the information needed to assess that dependence
is thinning — and that is a measured trend, not a complaint.

**Behaviour drifts under you, and versions retire.** A base model is not a stable dependency in the
sense a library is. Endpoints get updated, deprecated and withdrawn on the provider's schedule;
prompts tuned against one snapshot degrade against the next; a model your evaluation suite was
calibrated on can simply stop existing. The GPT-4o episode above is the visible version of a
constant, quieter one. Pin model versions explicitly, keep a regression suite you can rerun on a new
snapshot in an afternoon, and treat provider changelogs as production events.

**Legal exposure is inherited along with the weights.** Training-data provenance, licence terms,
personal data and output-ownership questions all originate above you and land on you. A downstream
developer typically cannot inspect the corpus, cannot prove what is in it, and cannot indemnify
themselves out of it beyond whatever the provider contractually offers — which is a different
posture from every other component in a normal software stack.

## Future Trends

**The regulatory threshold is becoming the operative definition of the base layer.** Article 51's
10²⁵ FLOP presumption gives "foundation model" something the 2021 definition deliberately avoided: a
number. The surrounding timeline is in motion — the European Commission tabled the Digital Omnibus
on **19 November 2025** and institutions reached provisional agreement on **6 May 2026**, deferring
Annex III high-risk obligations to **2 December 2027** — but the general-purpose model rules and the
compute threshold were not among the things postponed. Expect the compute line, not the CRFM
definition, to be what "foundation model" means in a contract.

**The base layer is redistributing faster than it is widening.** It is still roughly a dozen
organisations, but not the same dozen. Hugging Face's 2026 review found Chinese models taking the
plurality of downloads on the Hub at **41%**, with China passing the United States in both monthly
and cumulative downloads. Open-weight releases from Alibaba, DeepSeek and others move where the
dependency sits without reducing how concentrated it is — 113,000 derivatives of one family is
homogenization with a different flag on it, not less homogenization.

**Cheaper adaptation keeps pushing differentiation upward.** [Distillation](https://howaiworks.ai/glossary/knowledge-distillation),
[mixture-of-experts](https://howaiworks.ai/glossary/mixture-of-experts) routing and parameter-efficient tuning all make
the layer above the base cheaper to occupy, which increases the number of products per base model
rather than the number of bases. Every efficiency gain in adaptation makes the hourglass narrower at
the waist, not wider.

**Domain foundation models are the growth area, and they inherit the same structure.** Weather,
genomics, materials, robot control and clinical imaging are each converging on a few broad-data
models that everyone in the field adapts. That is a real gain — Evo 2 and AIFS are better than what
preceded them — and it reproduces the dependency exactly one field at a time. The question worth
asking about any of them is the one the 2021 report asked first: when this model is wrong, how many
systems are wrong with it?

## Frequently Asked Questions

### Is 'foundation model' just marketing for a big AI model?

No, though it is often used that way. The 2021 Stanford definition says nothing about size: a foundation model is one trained on broad data and adapted to a wide range of downstream tasks. Arc Institute's Evo 2 genome model has 40 billion parameters and is unambiguously a foundation model; a much larger model built and used for exactly one internal task is not. The criterion is what gets built on top of it.

### Who coined the term foundation model?

Stanford's Center for Research on Foundation Models, in a report posted to arXiv on 16 August 2021 with 114 authors. They chose a new word deliberately: 'large language model' was too narrow, and 'pre-trained model' described the training recipe rather than the role the model plays for everything downstream.

### What is the difference between a foundation model and a large language model?

An LLM is a foundation model whose data happens to be text. Foundation models also cover DNA, weather, protein structure, robot control and images. The overlap is large enough that people use the terms interchangeably, but the categories are defined on different axes: one by modality and architecture, the other by the fact that many downstream systems depend on it.

### What is homogenization and why is it a risk?

Homogenization is thousands of applications inheriting their behaviour from a handful of base models. The 2021 report put it plainly: 'the defects of the foundation model are inherited by all the adapted models downstream.' The same property that makes one good base model lift an entire industry makes one bad update, one bias, or one price change propagate everywhere at once.

### Does a foundation model have to cross the EU AI Act's 10^25 FLOP threshold?

No. That threshold marks presumed systemic risk for a general-purpose model, not the boundary of the category. Evo 2 cost roughly 2.2 × 10^24 FLOP to train — about a quarter of the line — and is still a foundation model. Frontier model and foundation model are different claims: one is about training compute, the other about how many things depend on you.

## Related

### Related terms

- [Pre-trained Models](https://howaiworks.ai/glossary/pre-trained-models)
- [Large Language Model (LLM)](https://howaiworks.ai/glossary/large-language-model)
- [Scaling Laws](https://howaiworks.ai/glossary/scaling-laws)
- [Transfer Learning (TL)](https://howaiworks.ai/glossary/transfer-learning)
- [Fine-tuning (FT)](https://howaiworks.ai/glossary/fine-tuning)
- [FLOPs (Floating Point Operations)](https://howaiworks.ai/glossary/flops)

---

Source: https://howaiworks.ai/glossary/foundation-models — HowAIWorks.ai