---
source: 'https://howaiworks.ai/glossary/precision-and-recall'
section: glossary
title: Precision and Recall
description: >-
  Precision, recall, F1 and the confusion matrix worked through one imbalanced
  example — and why the model with the best accuracy can catch nothing at all.
tags:
  - Machine Learning
  - model evaluation
  - metrics
  - classification
  - supervised learning
  - statistics
category: Machine Learning
datePublished: '2026-07-24'
lastUpdated: '2026-07-24'
---

# Precision and Recall

> Precision, recall, F1 and the confusion matrix worked through one imbalanced example — and why the model with the best accuracy can catch nothing at all.

## Definition

Precision and recall are the two halves of "how good is this model?" that plain accuracy throws
away. **Precision is the share of the things a model flagged that turned out to be real. Recall is
the share of the real things that the model flagged.** Precision's denominator is your own output;
recall's denominator is the world.

The reason they deserve a page rather than a footnote is one piece of arithmetic that surprises
almost everyone the first time. Take an end-of-line inspection system on a factory line: 50,000
parts go past the camera and 500 of them are genuinely defective. Tune the classifier to its
best **accuracy — 99.00%** — and it is exactly as accurate as a machine that stamps every part
"fine" without looking, while letting **three defects in five** reach the customer. Accuracy is a
weighted average across classes in which each class counts in proportion to how common it is, so
when 99% of the parts are good, accuracy is a measurement of good parts and nothing else. Precision
and recall do not have this defect, for a simple structural reason: neither of them counts the true
negatives at all.

Which of the two you should care about is not a matter of preference. It is set by which of your two
mistakes costs more, and the answer is usually lopsided by an order of magnitude or more. A cancer
screen's false negative is a tumour found a year later; its false positive is a follow-up scan.
A résumé filter's false positive is an hour of an interviewer's time; its false negative is a hire
you never knew you missed. Write down both prices before you choose a metric, because the metric you
choose will silently pick an operating point for you — and as the worked numbers below show, F1 and
accuracy pick the *most expensive* setting of the three on this inspection line.

## How It Works

### The four counts everything is made of

Every classification metric you have ever seen — accuracy, precision, recall, F1, specificity,
ROC-AUC — is arithmetic on four numbers. Run the inspection model over the 50,000 parts at its
default cutoff of 0.50. It flags 1,200 parts for re-inspection, and 300 of those flags are real
defects:

|                 | actually defective | actually good | total  |
|-----------------|-------------------:|--------------:|-------:|
| **flagged**     | 300 (TP)           | 900 (FP)      | 1,200  |
| **not flagged** | 200 (FN)           | 48,600 (TN)   | 48,800 |
| **total**       | 500                | 49,500        | 50,000 |

That table is the **confusion matrix**. The four cells sum to the test set, and the whole rest of
this page is division. True positives are defects caught; false positives are good parts pulled off
the line for nothing; false negatives are defects shipped; true negatives are the enormous, boring
mass of good parts correctly ignored.

Now read four metrics off the same four numbers:

**Precision** = TP / (TP + FP) = 300 / 1,200 = **0.250**. One flag in four is a real defect; the
re-inspection team throws away three quarters of its effort.

**Recall** = TP / (TP + FN) = 300 / 500 = **0.600**. Two defects in five are shipped to customers.

**Accuracy** = (TP + TN) / 50,000 = 48,900 / 50,000 = **0.978**.

**F1** = 2PR / (P + R) = 2 × 0.250 × 0.600 / 0.850 = **0.353**.

Notice the spread. The same model, on the same data, in the same minute, is either a 97.8% success
or a 25%-precision failure depending entirely on which division you perform. Nothing about the model
changed between those two sentences.

### Why accuracy is a liar here, in one line

Build the laziest possible competitor: a classifier that never flags anything. It has no camera and
no model. TP = 0, FP = 0, FN = 500, TN = 49,500, so its accuracy is 49,500 / 50,000 = **0.990** —
*higher* than the real model's 0.978. Its recall is 0 / 500 = **0.000**. Its precision is not low;
it is undefined, because it flagged nothing to be precise about.

This is the base-rate argument, and it is the reason the page exists. Any metric whose formula
contains TN inherits the size of the majority class, and on a 1% positive rate the majority class is
99 times louder than the thing you are trying to detect. Precision and recall are the two ratios
that omit TN, which is exactly why they survive imbalance. The same arithmetic gets worse as the
positive class gets rarer: at 1 in 10,000, doing nothing scores 99.99%, and the problem shades into
[anomaly detection](https://howaiworks.ai/glossary/anomaly-detection), where the queue rather than the model is the
subject.

### Which one do I care about? Put a price on each error

The honest answer to "precision or recall" is not a rule of thumb, it is a ratio. Suppose that on
this line a needlessly re-inspected part costs **$8** of an operator's time, and a defect that
escapes to a customer costs **$400** in warranty, shipping and goodwill. The ratio is 50 to 1, so
the rule follows immediately: **keep loosening the threshold until you are raising more than 50
false alarms for each additional defect you catch.** Past that point the extra inspections cost more
than the defects they prevent; before it, you are saving money by flagging more.

That single ratio dissolves most arguments about which metric to report. It is also the reason no
dataset can answer the question for you — the exchange rate between your two errors lives in your
business, not in your data, and two companies running the identical model will correctly choose
different cutoffs.

### Precision and recall belong to a threshold, not to a model

A classifier does not really emit a label. It emits a **score**, and a label appears only when
somebody picks a cutoff. Move the cutoff and both numbers move, in opposite directions, with no
retraining and no new data. Here is the same model at three cutoffs, plus the do-nothing baseline,
with the $8/$400 cost attached:

| cutoff       | TP  | FP    | FN  | precision | recall | F1    | accuracy | cost     |
|--------------|----:|------:|----:|----------:|-------:|------:|---------:|---------:|
| 0.20 (loose) | 450 | 3,550 |  50 | 0.113     | 0.900  | 0.200 | 0.9280   | $48,400  |
| 0.50         | 300 |   900 | 200 | 0.250     | 0.600  | 0.353 | 0.9780   | $87,200  |
| 0.80 (strict)| 200 |   200 | 300 | 0.500     | 0.400  | 0.444 | 0.9900   | $121,600 |
| flag nothing |   0 |     0 | 500 | undefined | 0.000  | —     | 0.9900   | $200,000 |

Three things in that table are worth more than the rest of this page.

First, **the strict cutoff's accuracy of 0.9900 is identical, to four decimal places, to the accuracy
of the model that flags nothing at all.** Not similar — equal. Accuracy cannot distinguish a
detector that catches 40% of defects from a detector that does not exist.

Second, **F1 and accuracy both rank the cutoffs in exactly the wrong order.** F1 rises 0.200 → 0.353
→ 0.444 as the threshold tightens; cost rises $48,400 → $87,200 → $121,600 alongside it. The
setting F1 likes best is the one that loses two and a half times as much money as the setting it
likes least. Check it against the exchange rate: going from cutoff 0.80 to 0.50 catches 100 more
defects for 700 more false alarms — 7 alarms per defect, comfortably under the break-even 50 — and
going from 0.50 to 0.20 catches 150 more for 2,650 more, or 17.7 per defect, still worth paying.

Third, none of this involved a better model. It is the same score column read four ways. "The model
isn't good enough" is usually the wrong diagnosis; "we are operating it at the wrong point" is
usually the right one, and it is free to fix.

### F1, and why it is a harmonic mean

Two numbers are awkward when you want to rank models on a leaderboard, so **F1** collapses them into
one: the harmonic mean of precision and recall, `F1 = 2PR / (P + R)`. At the default cutoff that is
0.353, against an ordinary average of (0.250 + 0.600) / 2 = 0.425. The harmonic mean sits below the
arithmetic mean always, and equals it only when P and R are identical.

The gap is the whole point, and it grows as the two numbers diverge. Consider a model that flags a
single part in the whole run and is right about it: precision 1.00, recall 0.002. The ordinary
average is a respectable-looking **0.501**. Its F1 is 2 × 1.00 × 0.002 / 1.002 = **0.004**. Because
the harmonic mean is the reciprocal of the average of the reciprocals, a small value contributes an
enormous reciprocal and dominates the result — so F1 refuses to let one excellent number cover for
one terrible one. That is precisely the behaviour you want from a single summary of two quantities
that can be traded against each other, since any metric that *could* be gamed by pushing one to 1.0
would be gamed exactly that way.

Where F1 is the wrong summary, the fix is usually **F-beta**, which the CoNLL-2003 shared task
writes as `Fβ = (β² + 1) · P · R / (β² · P + R)`. Setting β = 1 recovers F1; β = 2 weights recall
twice as heavily as precision, β = 0.5 does the reverse. On the inspection line, F2 scores the three
cutoffs 0.375 / 0.469 / 0.417 — it prefers the middle setting, whereas F1 preferred the strictest.
Changing β changed which model you would ship. Choosing β is the same decision as choosing the
threshold, made in a different notation, and it has the same answer: the relative cost of the two
errors.

### The same matrix, a different "positive"

Here is a property of F1 that surprises people who have used it for years: it is not symmetric.
Take the confusion matrix above, change nothing about the predictions, and merely decide that a
*good part* is the positive class. TP becomes 48,600, FP becomes 200, FN becomes 900, and:

- calling a defect the positive class: precision 0.250, recall 0.600, **F1 = 0.353**
- calling a good part the positive class: precision 0.996, recall 0.982, **F1 = 0.989**

Same model, same predictions, same test set, F1 nearly tripled. F1 ignores TN, so which cell counts
as "the big boring one" is a choice, and the metric changes drastically with it. Always say which
class is positive when you report an F1, and be suspicious of any leaderboard that does not. If you
want a single number that does not have this behaviour, the **Matthews correlation coefficient**
uses all four cells and returns **0.378** for this matrix whichever class you nominate.

### Curves, and where PR beats ROC

Rather than judge at one cutoff, sweep every cutoff and plot the result. The **precision-recall
curve** plots precision against recall; the **ROC curve** plots recall (also called the true
positive rate, or sensitivity) against the false positive rate FP / (FP + TN). The area under
either is a threshold-free summary of the ranking.

The trap is that the false positive rate's denominator is the entire negative class, which on
imbalanced data is enormous and forgiving. Read the loose cutoff both ways: 3,550 false alarms out
of 49,500 good parts is a false positive rate of **7.2%**, a point that looks strong on an ROC plot
and flatters the AUC. The same 3,550 false alarms against 450 catches is a precision of **11.3%** —
nearly nine parts pulled for every real defect. The PR curve shows the second reading immediately
because precision's denominator is what you flagged rather than what exists.

The baselines differ too, and this is the most practical way to remember which to use. A random
ranker always scores ROC-AUC 0.5, whatever the class balance. Its PR-AUC is the **base rate** —
0.01 here — so a PR-AUC of 0.30 on this data is a thirtyfold improvement over chance, while an
ROC-AUC of 0.85 tells you much less than it appears to. Davis and Goadrich showed at ICML 2006 that
the two spaces are not in conflict — one curve dominates another in ROC space if and only if it
dominates in PR space — so they cannot disagree about which model is better; they disagree about how
*large* the difference looks, and on skewed data PR is the space where the difference is visible.

### How much should you trust these two numbers?

Recall is estimated only from the positives in your test set, so its uncertainty is governed by how
many positives you have, not by how large the test set is. With 500 defects and a recall of 0.600,
the 95% interval is roughly ±4.3 percentage points. Cut the test set down to 50 defects — still
5,000 parts at this base rate — and the same estimate carries ±13.6 points, which is wide enough
that a model that genuinely improved recall from 0.60 to 0.70 would be indistinguishable from noise.
Precision is estimated from the flags instead: 1,200 of them here, giving about ±2.5 points.

The practical consequence is that on rare-positive problems your test set must be sized by the count
of positives you need, not by the fraction of data you feel like holding out — and that
[cross-validation](https://howaiworks.ai/glossary/cross-validation) folds must be stratified, or one fold will end up
with almost no positives and a recall estimate that is pure noise.

## Real-World Applications

**Two-stage retrieval in search and RAG.** Production
[retrieval-augmented generation](https://howaiworks.ai/glossary/retrieval-augmented-generation) systems split the job in
two precisely along this axis. Stage one — a
[vector search](https://howaiworks.ai/glossary/vector-search) or keyword index — is tuned for **recall@k**: fetch the top
100 or 500 candidates and make sure the right passage is somewhere in there, because nothing
downstream can recover a document that was never retrieved. Stage two, a cross-encoder reranker,
is tuned for **precision@10**, since the language model will only read a handful of passages and
each irrelevant one spends context and invites a wrong answer. Teams that report a single
"retrieval accuracy" for the pipeline cannot see which stage is failing; recall@100 and precision@10
are two different bug reports.

**Named-entity recognition.** The CoNLL-2003 shared task, still the standard NER benchmark,
publishes 23,499 entity mentions across 203,621 tokens in its English training split — roughly one
mention every nine tokens. A tagger that labels every token "not an entity" is therefore right about
the overwhelming majority of them, which is why the task specifies `Fβ=1` over exactly-matched
entities rather than token accuracy, and why every NER paper since reports precision, recall and F1
as a triple. The same logic carries into extractive question answering, where SQuAD scores token
overlap between the predicted and reference span with an F1 rather than demanding exact matches.

**Object detection.** The COCO benchmark's headline metric, AP, is literally the area under a
precision-recall curve — computed at 101 recall thresholds, averaged across 10 intersection-over-union
thresholds from .50 to .95 in steps of .05, and then across all 80 object categories. There is no
accuracy figure on the leaderboard, and there could not be: an image contains an unbounded number of
places where an object is not, so the true-negative count is undefined and every metric that needs
it is unavailable. [Computer vision](https://howaiworks.ai/glossary/computer-vision) had to become precision-and-recall
native, and the practice spread from there.

## Key Concepts

- **Precision has other names.** In clinical medicine, recall is *sensitivity*, precision is
  *positive predictive value*, and TN / (TN + FP) is *specificity*. The vocabularies matter because
  the field that says "PPV" also insists you state the prevalence next to it — sensitivity is a
  property of the test, PPV is a property of the test *and* the population, which is the base-rate
  point wearing different clothes.
- **Micro-averaging over classes is just accuracy.** Pool the TP, FP and FN counts across all
  classes of a single-label multi-class problem and micro-precision, micro-recall and accuracy all
  come out identical. Macro-averaging — compute per class, then average — is the one that lets a
  rare class pull the number down, which is usually what you wanted. Reporting either without
  saying which is a common way to mislead by accident.
- **Precision@k for ranked output.** When a system produces a ranked list and a human reviews a
  fixed number of items per shift, the operational metric is precision within that budget, not at a
  threshold. Twenty-item queues and score cutoffs are different products.
- **Calibration is a separate question.** A model can rank perfectly — every positive scored above
  every negative, ROC-AUC 1.0 — while its scores are nowhere near probabilities. The moment you
  multiply a score by a cost, as the $8/$400 rule does, you need the score to mean what it says.

## Challenges

**You usually cannot measure recall, only estimate it.** Precision is directly observable: read your
flag queue and count. Recall needs the positives you missed, which by construction you did not see.
The standard fix is to audit a random sample of the *unflagged* pool, and the arithmetic is
discouraging. At the default cutoff, 48,800 parts go unflagged and 200 of them are defective — 0.41%.
Audit 2,000 of them and you expect about 8 defects, whose 95% interval runs from roughly 3 to 16;
scaled back up, the missed-defect count lands somewhere between 84 and 385, so your measured recall
is somewhere between **44% and 78%**. That is a range wide enough to contain both "acceptable" and
"fire the vendor", from an audit ten times larger than most teams run.

**Precision does not transfer between datasets, and neither does F1.** Recall is a property of the
model on the positive class alone, so it is roughly stable if the positives look the same. Precision
depends on how many negatives are in the pool, so the identical model scores 25% precision on this
line and something quite different on a line with a 3% defect rate. Comparing your F1 to a published
F1 from another dataset is therefore close to meaningless unless the base rates match, and they
almost never do. Compare against your own base rate and your own previous release.

**Choosing the threshold on the test set is a form of [overfitting](https://howaiworks.ai/glossary/overfitting).** Picking
the cutoff that maximises F1 and then reporting that F1 on the same data leaks the answer into the
result, and it is extremely common because the sweep is so easy to run. Pick the operating point on a
validation split, then report on data that neither the model nor the threshold has seen.

**The base rate moves and takes precision with it.** A cutoff chosen when the defect rate was 1% is
silently wrong when a supplier change pushes it to 3%: recall may hold, but precision will jump and
the review team's workload with it, and nobody will be alerted because accuracy barely twitches.
Precision and recall are [monitored](https://howaiworks.ai/glossary/monitoring) quantities, not launch-day numbers, and
the base rate belongs on the same dashboard.

**A single number hides who is failing.** An aggregate recall of 0.600 can be 0.85 on one product
line and 0.20 on another, or — where the classifier makes decisions about people — 0.85 for one
demographic group and 0.20 for another. Per-segment confusion matrices are the minimum honest
reporting for any deployed [classification](https://howaiworks.ai/glossary/classification) system.

## Code Example

Every number on this page comes out of four counts and some division. This runs the whole argument,
including the do-nothing baseline and the class-swap demonstration:

```python
# 50,000 inspected parts, 500 of them genuinely defective: a 1% base rate.
# One model, one set of scores, three cutoffs, plus the do-nothing baseline.
# Counts are (true positives, false positives, false negatives, true negatives).
points = {
    "cutoff 0.20": (450, 3550, 50, 45950),
    "cutoff 0.50": (300, 900, 200, 48600),
    "cutoff 0.80": (200, 200, 300, 49300),
    "flag nothing": (0, 0, 500, 49500),
}

def pr(tp, fp, fn):
    precision = tp / (tp + fp) if tp + fp else float("nan")
    return precision, tp / (tp + fn)

def fbeta(precision, recall, beta):
    b2 = beta ** 2
    return (1 + b2) * precision * recall / (b2 * precision + recall)

for label, (tp, fp, fn, tn) in points.items():
    precision, recall = pr(tp, fp, fn)
    accuracy = (tp + tn) / (tp + fp + fn + tn)
    cost = 8 * fp + 400 * fn    # $8 per needless re-inspection, $400 per escaped defect
    print(f"{label:<13} P {precision:5.3f}  R {recall:5.3f}"
          f"  F1 {fbeta(precision, recall, 1):5.3f}"
          f"  F2 {fbeta(precision, recall, 2):5.3f}"
          f"  acc {accuracy:.4f}  cost ${cost:>7,}")

# Same 50,000 parts, same predictions - only the word "positive" moves.
tp, fp, fn, tn = points["cutoff 0.50"]
print()
for label, counts in [("defect = positive", (tp, fp, fn)), ("good part = positive", (tn, fn, fp))]:
    precision, recall = pr(*counts)
    print(f"{label:<21} P {precision:.3f}  R {recall:.3f}  F1 {fbeta(precision, recall, 1):.3f}")
```

```
cutoff 0.20   P 0.113  R 0.900  F1 0.200  F2 0.375  acc 0.9280  cost $ 48,400
cutoff 0.50   P 0.250  R 0.600  F1 0.353  F2 0.469  acc 0.9780  cost $ 87,200
cutoff 0.80   P 0.500  R 0.400  F1 0.444  F2 0.417  acc 0.9900  cost $121,600
flag nothing  P   nan  R 0.000  F1   nan  F2   nan  acc 0.9900  cost $200,000

defect = positive     P 0.250  R 0.600  F1 0.353
good part = positive  P 0.996  R 0.982  F1 0.989
```

Read the columns against each other rather than down. Accuracy is monotone in the threshold and
peaks at 0.9900, tying a classifier with no model in it. F1 is monotone the same way and picks the
$121,600 setting. F2 picks the $87,200 one. Cost — the only column that knows what the errors are
worth — picks the $48,400 one, the row where precision looks worst. And the last two lines are the
identical prediction set scored twice, with F1 moving from 0.353 to 0.989 because someone changed
which class was called positive.

The lesson is not that F1 is bad. It is that a summary metric is a guess at your cost function, and
when you know the actual costs you should use them and treat precision and recall as the two
quantities the guess was summarising.

## Frequently Asked Questions

### What is the difference between precision and recall?

Precision asks: of everything I flagged, how much was real? Recall asks: of everything real, how much did I flag? Precision's denominator is your own output, recall's is the world. A model that flags one item and gets it right has 100% precision and almost no recall; a model that flags everything has 100% recall and precision equal to the base rate.

### Which matters more, precision or recall?

Whichever error costs more. Write down the price of one false positive and one false negative, take the ratio, and that ratio tells you how many false alarms you should be willing to accept to catch one more real case. If a missed defect costs $400 and a needless re-inspection costs $8, you should keep loosening the threshold until you are raising 50 false alarms per extra defect caught.

### Why is F1 the harmonic mean rather than the average?

Because the harmonic mean is dragged down by the smaller of the two numbers, and the ordinary average is not. A model that flags one part out of 50,000 and is right about it — precision 1.00, recall 0.002 — has an ordinary average of 0.501 and an F1 of 0.004. F1 refuses to let one good number cover for one bad one.

### Should I use PR-AUC or ROC-AUC?

PR-AUC when positives are rare. The false positive rate that ROC plots has the whole negative class in its denominator, so on 1% positives 3,550 false alarms is a false positive rate of 7.2% — a respectable-looking point that corresponds to 11% precision. A precision-recall curve shows that immediately. Their random baselines also differ: 0.5 for ROC-AUC always, the base rate for PR-AUC.

### What is a good precision or recall score?

There is no threshold that makes a number good, because both depend on the base rate of your data and on the cutoff you chose. Precision of 25% is dismal for a spam filter and excellent for an alert queue at a 1-in-10,000 event rate. Compare against the base rate and against the cost of the two errors, never against a number from someone else's dataset.

### How do I improve recall without destroying precision?

Lowering the threshold trades one directly for the other and is free, so it is not really an improvement — it is a move along the curve you already have. Genuine gains come from a better score: more or cleaner labels for the rare class, features that separate it, or class weighting in the loss. Measure the change on the precision-recall curve, not at one cutoff, or you cannot tell which of the two you did.

## Related

### Related terms

- [Classification (CLF)](https://howaiworks.ai/glossary/classification)
- [Anomaly Detection (AD)](https://howaiworks.ai/glossary/anomaly-detection)
- [Cross-Validation (CV)](https://howaiworks.ai/glossary/cross-validation)
- [Logistic Regression (LR)](https://howaiworks.ai/glossary/logistic-regression)
- [Information Retrieval (IR)](https://howaiworks.ai/glossary/information-retrieval)
- [Benchmark](https://howaiworks.ai/glossary/benchmark)

---

Source: https://howaiworks.ai/glossary/precision-and-recall — HowAIWorks.ai