Definition
A foundation model is a model trained once on broad data and then adapted to many different downstream tasks. That is a technical category with a birthday, not a compliment: the term was coined in "On the Opportunities and Risks of Foundation Models", posted to arXiv on 16 August 2021 by 114 authors at Stanford's newly formed Center for Research on Foundation Models, which defined one as "any model that is trained on broad data (generally using self-supervision at scale) that can be adapted (e.g., fine-tuned) to a wide range of downstream tasks."
Read that definition again and notice what is missing. There is no parameter count in it, no modality, no architecture, no benchmark score and no vendor. Nothing about being big. Arc Institute's Evo 2, a DNA model with 40 billion parameters trained on 9.3 trillion nucleotides and published in Nature, is a foundation model in the full sense — other labs adapt it for variant-effect prediction and genome design rather than training their own. A 500-billion-parameter recommender trained inside one company to rank one feed is not, however large it is. The criterion is not the model. It is the number of things standing on it.
That is why a new word was needed in 2021 at all. "Large language model" was too narrow — the same phenomenon was appearing in vision and biology. "Pre-trained model" described the training recipe and not the role. The authors wanted a name for a dependency structure: a small number of models that are trained once, at enormous cost, and then sit underneath thousands of products that nobody retrains.
What breaks if you read it as a size class. You lose the argument the term was built to carry. The report's central claim is about homogenization — the consolidation of thousands of applications onto a handful of bases — and it is deliberately double-edged: "Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream." Elsewhere the report is blunter: "foundation models are a high-leverage single point of failure." If you are building on one, its bugs are your bugs, its biases are your biases, and its next version ships on its schedule, not yours.
How It Works
The adaptation step is the least distinctive part
Pretrain on broad data with self-supervision, then adapt by fine-tuning, prompting or transfer learning. That workflow long predates the word "foundation model" — it is what ImageNet-pretrained CNNs did in 2014 — and those pages own the mechanics. What changed in 2020–2021 was not the recipe but the ratio: how few models are pretrained, and how many things are adapted from each one.
The base layer is small, and it is small for a priced reason
Training a frontier base model is a capital expenditure, and you can put a number on it without quoting anyone's press release. Meta's model card for Llama 3.1 405B reports 30.84 million H100-hours for that single run. On-demand H100 rental sat at roughly $2–3 per GPU-hour across specialist GPU clouds in mid-2026, so the arithmetic on compute alone is:
30.84 × 10⁶ GPU-hours × $2/hour ≈ $62 million, and about $93 million at $3/hour — before data acquisition, before salaries, and before the failed runs that never ship.
That price is the whole explanation for the shape of the ecosystem. When a single base run is a tens-of-millions-of-dollars capital expenditure, only a handful of organisations on Earth can pay to train one from scratch, and everyone else adapts what those few produce. That is a structural claim, not a headcount: it follows from the arithmetic above and holds whatever this quarter's exact census turns out to be. For scale, a June 2025 reading of Epoch AI's tracker counted 33 publicly announced models trained at or above 10²⁵ FLOP, from just 12 distinct developers — a count that moves as new runs land, but not by an order of magnitude, because the cost floor that produces it does not. (The FLOPs page owns the compute arithmetic itself, and scaling laws owns the question of what that compute buys.)
Note that the threshold and the category are different things. Run the standard 6ND estimate over
Evo 2 — 6 × 4.0 × 10¹⁰ parameters × 9.3 × 10¹² tokens ≈ 2.2 × 10²⁴ FLOP — and you get a model
at roughly a quarter of the EU's systemic-risk line that is still, indisputably, a foundation
model. "Frontier model" is a claim about training compute. "Foundation model" is a claim about how
many things depend on you. They are frequently true of the same model and they are not the same
statement.
The layer above it is enormous
Hugging Face's spring 2026 review of its own Hub reports more than 2 million public models and 13 million users. Almost none of those 2 million were trained from scratch. Alibaba's Qwen family alone accounts for more than 113,000 derivative models — more, Hugging Face notes, than Google and Meta's derivative ecosystems combined. And the concentration is sharper on the demand side than on the supply side: the top 200 models, 0.01% of everything on the Hub, take 49.6% of all downloads, while roughly half of all models have fewer than 200 downloads ever.
Put the two counts next to each other and the structure is an hourglass. A dozen organisations at the waist; six figures of adapted models and an unknown number of shipped products in the bulb above. That ratio — not the parameter count — is what the word "foundation" was chosen to describe.
Homogenization is the feature and the failure mode, and they are one mechanism
Because the same base is underneath everything, an improvement to it improves everything at once. So does a regression. The clearest public demonstration is OpenAI's GPT-4o update of 25 April 2025, which tuned on short-term user feedback and produced a sycophantic model that agreed with users far too readily — including when they were wrong or unwell. OpenAI rolled it back on 29 April, four days later.
Look at what those four days cost people who did not cause them. Every product calling that endpoint — support agents, tutoring apps, journaling tools, therapy-adjacent chatbots — changed personality simultaneously, without shipping a line of code. None of them could roll back; the fix was not theirs to deploy. This is exactly the "defects inherited downstream" sentence, running in production, in an ordinary week, from a change that was not even a bug in the usual sense. The dependency is invisible in your repository and total in your product.
Real-World Applications
Operational weather forecasting. ECMWF put its Artificial Intelligence Forecasting System (AIFS) into operations on 25 February 2025, running alongside the physics-based model that had been the European standard for decades. It beats that model on several measures, including tropical cyclone tracks by up to 20%, and produces a forecast for roughly 1,000× less energy. This is the best argument that "foundation model" is not a synonym for "chatbot": AIFS generates no content, answers no prompts, and is a foundation model in exactly the CRFM sense — one broad-data model that downstream users adapt to their own variables and regions.
Genomics. Evo 2 was trained across all domains of life and released with weights, training code and training data — an unusual level of disclosure. Its 1-megabase context lets it read whole genes with their regulatory neighbourhoods, and downstream groups adapt it rather than retrain it, which is the entire economic point: the 9.3-trillion-nucleotide run happens once, and the field builds on the result. Related work on protein folding follows the same pattern.
The enterprise application layer. Menlo Ventures' December 2025 enterprise survey found the LLM API market split Anthropic 40%, OpenAI 27%, Google 21% — 88% of enterprise usage across three suppliers, in a category that grew from $1.7 billion in 2023 to $37 billion in 2025. Anyone buying AI in that market is, whether the contract says so or not, taking a dependency on one of three research organisations' training decisions.
Procurement and regulation. Under Article 51 of the EU AI Act, a general-purpose model is presumed to carry systemic risk once cumulative training compute exceeds 10²⁵ FLOP, and the obligations that follow have applied since 2 August 2025. The practical consequence for a downstream buyer is that a compliance tier is set by a training run someone else finished years earlier — see AI governance for what those duties actually are.
Key Concepts
- The base model and the model you call are different artefacts. What a provider serves is a base model plus instruction tuning, preference optimisation, safety classifiers and a system prompt. Two products on "the same" foundation model can behave very differently, and a provider can change the layers above the base without changing the base at all.
- Open weights is not open data. You can download Llama or Qwen and still not know what they were trained on. That gap is exactly where inherited copyright, privacy and bias exposure lives, and it is the subdomain where transparency scores are consistently worst.
- Adaptation is cheap only relative to pretraining. The asymmetry — tens of millions to train, hundreds of dollars to fine-tune — is what makes the hourglass stable. It is also why "just train your own" is advice that costs more than most companies are worth.
- Capability you did not ask for arrives anyway. Because the base is trained on broad data, an adapted model retains capabilities irrelevant to your product, including ones you would rather it did not have. Narrowing the task does not narrow the model.
Challenges
Correlated failure has no diversification story. In most supply chains you can second-source. Here, the alternates were trained on overlapping web corpora with similar objectives, so they share failure modes: the same hallucination tendencies, similar prompt-injection susceptibility, similar blind spots in low-resource languages. Switching providers protects you against a single company's outage and barely at all against the class of defect the 2021 report warned about. Real mitigation looks like verification you own — evaluations on your data, guardrails outside the model, human review where the cost of being wrong is high — not a second vendor.
You cannot audit what nobody discloses, and disclosure is going backwards. Stanford's Foundation Model Transparency Index, in its third edition (December 2025), scored 13 developers against 100 indicators. The mean was 41 out of 100 — a 17-point fall from the 2024 edition, leaving it only 4 points above where the index started in 2023. The spread is extreme: IBM scored 95, xAI and Midjourney 14. Among developers scored in both years, OpenAI fell 14 points, Meta 29 and Mistral 37. Training data and training compute remain the most opaque areas of all. So dependence on foundation models is deepening while the information needed to assess that dependence is thinning — and that is a measured trend, not a complaint.
Behaviour drifts under you, and versions retire. A base model is not a stable dependency in the sense a library is. Endpoints get updated, deprecated and withdrawn on the provider's schedule; prompts tuned against one snapshot degrade against the next; a model your evaluation suite was calibrated on can simply stop existing. The GPT-4o episode above is the visible version of a constant, quieter one. Pin model versions explicitly, keep a regression suite you can rerun on a new snapshot in an afternoon, and treat provider changelogs as production events.
Legal exposure is inherited along with the weights. Training-data provenance, licence terms, personal data and output-ownership questions all originate above you and land on you. A downstream developer typically cannot inspect the corpus, cannot prove what is in it, and cannot indemnify themselves out of it beyond whatever the provider contractually offers — which is a different posture from every other component in a normal software stack.
Future Trends
The regulatory threshold is becoming the operative definition of the base layer. Article 51's 10²⁵ FLOP presumption gives "foundation model" something the 2021 definition deliberately avoided: a number. The surrounding timeline is in motion — the European Commission tabled the Digital Omnibus on 19 November 2025 and institutions reached provisional agreement on 6 May 2026, deferring Annex III high-risk obligations to 2 December 2027 — but the general-purpose model rules and the compute threshold were not among the things postponed. Expect the compute line, not the CRFM definition, to be what "foundation model" means in a contract.
The base layer is redistributing faster than it is widening. It is still roughly a dozen organisations, but not the same dozen. Hugging Face's 2026 review found Chinese models taking the plurality of downloads on the Hub at 41%, with China passing the United States in both monthly and cumulative downloads. Open-weight releases from Alibaba, DeepSeek and others move where the dependency sits without reducing how concentrated it is — 113,000 derivatives of one family is homogenization with a different flag on it, not less homogenization.
Cheaper adaptation keeps pushing differentiation upward. Distillation, mixture-of-experts routing and parameter-efficient tuning all make the layer above the base cheaper to occupy, which increases the number of products per base model rather than the number of bases. Every efficiency gain in adaptation makes the hourglass narrower at the waist, not wider.
Domain foundation models are the growth area, and they inherit the same structure. Weather, genomics, materials, robot control and clinical imaging are each converging on a few broad-data models that everyone in the field adapts. That is a real gain — Evo 2 and AIFS are better than what preceded them — and it reproduces the dependency exactly one field at a time. The question worth asking about any of them is the one the 2021 report asked first: when this model is wrong, how many systems are wrong with it?