Definition
A model card is a short, standardized document that ships alongside a released machine learning model and states, in one place, four things: what the model is meant to be used for, what data it was trained on at a high level, how well it performs — broken down by group, not just on average — and what its known limitations are. It is to a model roughly what a nutrition label is to a food product: not the recipe, but the disclosures a consumer needs to decide whether to use it and how.
The format was proposed by Margaret Mitchell and colleagues in "Model Cards for Model Reporting" (Mitchell et al., presented at the FAT* conference on Fairness, Accountability, and Transparency in 2019). The paper's argument is narrower and sharper than "models should be documented." Its claim is that the single most important thing a model report can do is refuse to summarize performance as one number. A model card's defining feature — the part that distinguishes it from a README or a marketing page — is that its evaluation section is disaggregated: performance is reported per group and per intersection of groups, because an aggregate score is an average, and an average hides the people the model works worst for.
That is the whole reason the artifact exists, and it is worth stating plainly because it is easy to miss. If a reader takes away only one sentence, it should be this: a model card is where the aggregate number is broken apart. Ship a model without one and every downstream user has to reverse-engineer facts you already knew — the population it was tested on, the group it fails for, the use it was never meant for — usually by discovering them in production, on real people.
How It Works
The paper proposes a fixed set of sections so that cards are comparable across models rather than each being a bespoke essay. The nine top-level sections are:
- Model Details — who built it, when, the version, the architecture, and how to cite it.
- Intended Use — the primary use cases, the intended users, and, critically, the out-of-scope uses the authors explicitly do not vouch for.
- Factors — the groups and conditions performance will be reported against (for example demographic groups, phenotypic groups such as Fitzpatrick skin type, or environmental conditions like lighting).
- Metrics — which measures are reported and why, including decision thresholds and confidence intervals.
- Evaluation Data — the dataset(s) the reported numbers come from, and why they were chosen.
- Training Data — mirrored to Evaluation Data where possible. The paper notes this section often cannot be provided in full in practice, so at minimum the card should describe the data at a high level.
- Quantitative Analyses — the disaggregated results.
- Ethical Considerations — foreseeable harms and the risk factors behind them.
- Caveats and Recommendations — everything that did not fit, including what still needs testing.
Most of these are ordinary documentation. The one that carries the paper's argument is Quantitative Analyses. In the authors' words, "quantitative analyses should be disaggregated, that is, broken down by the chosen factors," and they should report two kinds of result: unitary results (how the model did on each factor on its own — say, by skin type, and separately by gender) and intersectional results (how it did on the combination — darker-skinned women, not "darker-skinned people" and "women" reported apart). Intersectional reporting is the part people skip, and it is the part that catches the worst failures.
Why the aggregate lies: a worked example
Suppose you evaluate a face-attribute classifier on a test set that is 90% lighter-skinned faces and 10% darker-skinned faces — a realistic imbalance, because that is how many benchmark sets were built. Say the model scores 99% accuracy on the majority group and 65% on the minority. The single number you would put on a slide is the weighted average:
0.90 × 99% + 0.10 × 65% = 89.1% + 6.5% = 95.6%
A 95.6% headline looks like a strong model. It is also a model that is wrong more than a third of the time for one in ten of the people who use it — and nothing in the aggregate reveals that. Change the test-set mix to 95/5 and the same 65% subgroup lifts the headline to 97.3%, hiding the failure better. The disaggregated table would have shown the 65% cell immediately; the aggregate was designed, structurally, not to.
This is not hypothetical. The study that made the case, Gender Shades (Buolamwini & Gebru, 2018) — the paper whose author, Timnit Gebru, is also a co-author of the model cards paper — audited commercial gender classifiers from IBM, Microsoft, and Face++ and reported error rates of up to 34.7% for darker-skinned women against a maximum of 0.8% for lighter-skinned men. Every one of those systems could have advertised high overall accuracy. The 34.7% cell only appears when you break the results down by skin type and gender at once, which is exactly what a model card's Quantitative Analyses section is built to force.
Real-World Applications
Model cards moved from a 2019 proposal to routine practice faster than most documentation standards. Three concrete forms of that are worth naming.
Hugging Face model cards. On the Hugging Face Hub, the model card is the model's README.md, with a block of structured YAML metadata (license, language, datasets, evaluation results) at the top and prose below. The Hub prompts authors to create one and can scaffold a template automatically, which is why nearly every model on the platform ships with at least a card stub. This is the mechanism that made "does it have a model card?" a normal question to ask about a model at all.
System cards. As released "models" became released systems — a base model plus safety filters, a usage policy, and tool access — several major labs, including OpenAI, Google, and Anthropic, began publishing system cards at launch: documents describing a system's evaluations, red-teaming, and limitations. A system card is the model card idea scaled up from a component to the assembled product, and transparency practices at the frontier now largely run through them.
Regulation-adjacent documentation. Model cards remain a voluntary community standard, but the regulatory floor beneath them is rising. The EU AI Act (Regulation 2024/1689) requires providers of high-risk AI systems to draw up technical documentation before the system is placed on the market or put into service — Article 11, with the minimum contents specified in Annex IV — covering intended purpose, performance metrics, and foreseeable risks. That is not a model card by name, and the legal document is far more extensive, but it demands much of the same disclosure a good card already contains, which is why teams increasingly treat a rigorous model card as the seed of their compliance file. (These high-risk obligations are dated: the Act entered into force on 1 August 2024, with the high-risk documentation requirements phasing in over the following years, so treat any specific compliance deadline as a figure to re-check against the current text.) This is one thread of the broader shift toward formal AI governance and enforceable accountability.
Key Concepts
A model card is not a datasheet. This is the distinction people conflate most, and the two are complements, not synonyms. A datasheet — from "Datasheets for Datasets" (Gebru et al., 2018) — documents the dataset: its motivation, composition, collection process, and recommended uses, by explicit analogy to the datasheet that accompanies every electronic component in the hardware industry. A model card documents the model. The dataset is an input; the model is the artifact you trained on it. A single model card may reference several datasets, each of which ideally has its own datasheet, and a good card's Training Data and Evaluation Data sections are effectively pointers into those datasheets. Ask "what is in this data?" and you want a datasheet; ask "how does this trained thing behave, and for whom?" and you want a model card.
Training data is disclosed at a high level, and often only that. The model cards paper is candid that full training-data disclosure "may not be possible in practice." Some of that is commercial secrecy, but part is genuine: publishing the exact training set can enable privacy attacks such as membership inference, where an adversary probes the model to determine whether a specific person's record was in its training data. So a card typically describes the data's sources, scale, and known gaps rather than enumerating it — which is a real limit on how much a card can tell you, not a loophole.
A model card is a transparency artifact, and transparency is not explainability. The card makes a model's behavior and provenance visible; it does not make any individual prediction understandable. A model can ship with an exemplary card and still be a black box on any given input — that is the province of explainable AI, a separate problem. The card's job is to tell you the model is 65% accurate for a group, not to tell you why it got a particular face wrong.
Challenges
The author fills it in, so the unflattering result can quietly not appear. A model card is self-reported. If the team never evaluates on darker-skinned faces, the Gender Shades cell simply does not exist in the table, and the card looks complete while omitting the number that mattered. Disaggregation only surfaces a gap along factors someone chose to measure; the groups left out of the Factors section are invisible by construction. This is the specific failure mode to watch for — not a card that is wrong, but a card whose evaluation set was quietly chosen to have no bad cells.
Auto-generated cards can be empty theatre. The same tooling that made cards ubiquitous also made it easy to ship a card that is all metadata and no evaluation — a filled-in template with the license and language tags populated and the Quantitative Analyses section left as boilerplate. A stub card gives the appearance of documentation, which can be worse than none, because it passes a "has a model card?" checkbox while disclosing nothing a reader could act on.
Cards go stale, especially behind an API. A model card describes the version it was written for. When a model is updated behind a stable API endpoint — a routine occurrence — the deployed behavior can drift away from the card while the card still reads as current. Any performance number on a card is a perishable fact with an implicit "as of" date; a card without version and date stamps on its results is describing a model that may no longer exist. Treat the numbers on a card the way you would treat a benchmark score without its date: as a historical note until proven otherwise.
Comparability is only as good as the shared benchmark. Two cards report accuracy, but on different evaluation sets, the numbers are not comparable — the same trap that makes a benchmark score meaningless without knowing which benchmark and which version produced it. The card format standardizes the sections; it cannot by itself standardize the measuring stick, so cross-model comparison still requires that both were evaluated the same way.