---
source: 'https://howaiworks.ai/glossary/monitoring'
section: glossary
title: Monitoring
description: >-
  Why a deployed model rots while every dashboard stays green — and what you can
  actually measure when the true labels arrive weeks late, or never.
tags:
  - monitoring
  - MLOps
  - concept drift
  - data quality
  - Machine Learning
category: Machine Learning
datePublished: '2025-08-20'
lastUpdated: '2026-07-22'
---

# Monitoring

> Why a deployed model rots while every dashboard stays green — and what you can actually measure when the true labels arrive weeks late, or never.

## Definition

**Monitoring a machine learning system is the practice of watching a program whose code has not
changed for evidence that the world it was trained on has.** That is the whole difference from
ordinary application monitoring, and it produces a failure mode normal observability is
architecturally incapable of seeing: a model that returns HTTP 200 well inside its latency budget,
passes every unit test, keeps its error rate at zero — and is quietly wrong more often each week.

The asymmetry that organises everything else is this: **you can see the inputs move today, and you
cannot see the accuracy fall until the labels arrive.** For a card-fraud model, the label is a
chargeback, and US billing-error rules give a cardholder 60 days from the statement to file one
([12 CFR 1026.13(b)(1)](https://www.ecfr.gov/current/title-12/section-1026.13)). On a calendar-month
billing cycle, a transaction on 22 July lands on a statement transmitted 1 August, and the dispute
window closes 30 September — so
whether today's predictions were right is a fact you will learn in **70 days**. Every technique in
this page is a way of coping with that gap.

## How It Works

### Application monitoring always gets an answer; model monitoring usually does not

A web service tells you when it fails. The request returns a 500, or times out, or the queue backs
up. The measurement is cheap and immediate because success is a property of the response itself.

A model returns `0.83`. Nothing in that response says whether `0.83` was the right answer, and
nothing ever will until reality reports back. So monitoring splits into two layers that get confused
constantly:

- **The serving layer** — latency, throughput, GPU utilisation, error rates, queue depth. This is
  ordinary [inference](https://howaiworks.ai/glossary/inference) infrastructure monitoring, it is a solved problem, and
  Prometheus and Grafana handle it. It tells you nothing about correctness.
- **The model layer** — is this thing still right? No standard web stack measures this, and in
  production you usually cannot measure it directly at all.

The rest of monitoring is about the second layer.

### Three things drift, and only two of them are visible

The vendor taxonomies of "monitoring types" are mostly marketing, but one distinction is real and
mechanical, because it is a statement about which probability distribution moved.

**Data drift** (also called covariate shift) is a change in `P(x)`, the distribution of the inputs.
Say a credit model built on applicants with a median income of £34,000 starts receiving applicants
at £48,000. **You can detect this today, from the requests alone, with no labels whatsoever** — which
is exactly why input monitoring is the backbone of every practical setup.

**Prediction drift** is a change in `P(ŷ)`, the distribution of the model's own output scores. Also
observable immediately. It is partly downstream of data drift, but it earns its own place because it
catches things per-feature checks miss: forty features can each shift within tolerance while their
*combination* pushes the score distribution somewhere new.

**Concept drift** is a change in `P(y|x)` — the relationship between input and correct answer. The
inputs can be pixel-for-pixel identical to last year's and the right answer has changed anyway.
Imagine a delivery-time model after a new regional warehouse opens: the orders arriving look exactly
as they always did, so every input monitor stays flat, while the true delivery time for those same
inputs has dropped by a day, so every estimate the model produces is now a day too long. **Concept drift
cannot be detected from inputs. Ever.** It is the drift that costs money, and it is invisible.

That asymmetry — cheap detection for the harmless case, no detection for the expensive one — is why
ML monitoring is a discipline rather than a dashboard.

### Measuring a shift you can actually see

The standard instrument for input drift is the **Population Stability Index (PSI)**. Bin a feature
using the training data's own quantiles, then compare the share of traffic in each bin now against
the share then:

```
PSI = Σ (aᵢ − eᵢ) · ln(aᵢ / eᵢ)
```

Do one by hand. Cut transaction amount at the training quartiles, so the reference is 25% per bin by
construction. Tuesday's traffic comes in at 18% / 22% / 28% / 32% — visibly lopsided on a bar chart.
Each term is a difference times a log ratio: `(0.18−0.25)·ln(0.72) = 0.0230`, then `0.0038`,
`0.0034`, `0.0173`. **PSI = 0.048.** Under every conventional threshold. The chart looked alarming
and the statistic says nothing has happened.

Friday comes in at 10% / 15% / 30% / 45%, and the same four terms sum to **0.315**. That is a real
shift, and PSI is far less twitchy than the human eye — which is most of the reason to use it.

It is worth knowing what PSI *is*, because the name hides it: expand the formula and it is exactly
`KL(a‖e) + KL(e‖a)`, the symmetrised Kullback–Leibler divergence, measured in nats. For the Friday
window those two halves are 0.1509 and 0.1643, and they sum to 0.3152. Nothing exotic is happening —
"population stability index" is credit-scoring vocabulary for an information-theoretic quantity that
predates it by decades.

### The number nobody tells you: what PSI reads when nothing is wrong

Almost every article quotes the same rule of thumb — below 0.10 "little change", 0.10 to 0.25
"moderate change", 0.25 and above "significant change, action required". Yurdakul and Naranjo, in
[Statistical properties of the population stability index](https://files.wmich.edu/s3fs-public/attachments/u730/2022/PSIfinal.pdf)
(*Journal of Risk Model Validation*, 2020), quote that rule and then say the quiet part: "These
benchmarks are used without reference to statistical type I or type II error rates."

They then derive the property that matters operationally. If the two samples really are drawn from
the same distribution, `(1/n + 1/m)⁻¹ · PSI` is approximately chi-square with `B − 1` degrees of
freedom, where `B` is the number of bins. Rearranged, the **expected PSI between two windows when
nothing has changed** is roughly

```
E[PSI] ≈ (B − 1) · (1/n + 1/m)
```

Put numbers in it. Ten bins, two windows of 1,000 rows each: `9 × 0.002 = 0.018`. Fine. Ten bins,
two windows of **200 rows** each: `9 × 0.01 = 0.090` — which is essentially the 0.10 "moderate
change" line, produced entirely by sampling noise, on data that has not moved at all. Simulate it and
the theory holds to three digits — the code example below does exactly that. A daily drift check on a
low-traffic segment, using the textbook threshold, is not monitoring; it is a random number generator
wired to a pager.

The fix is not a better threshold, it is a threshold derived from your own window size. Comparing
against a large, fixed reference set rather than against another small window drops the reference's
term out of `1/n + 1/m` and halves the noise floor; setting the alert at the 99th percentile of the
chi-square then gives you a false-alarm rate you chose on purpose rather than inherited from a blog
post.

### Coping with labels that arrive late, or never

Return to the fraud model. Retraining on 1 September, the most recent month whose chargebacks have
fully settled is June — July's dispute window does not close until 30 September. So the "accuracy"
number on the dashboard on 1 September describes a model scoring June's traffic, and the decision to
retrain is being made on **zero percent** of the last two months' labels.

Partial labels are available sooner, and they are treacherous rather than useless: the disputes that
arrive in week one are not a random sample of the disputes that will eventually arrive. Large
fraudulent charges get noticed faster than small ones, so an early-window "precision" is measured on
a biased subset and reads systematically better than the truth. Treat it as a leading indicator with
a known sign of error, not as the metric.

Three practices fill the gap, and all three are label-free:

- **Watch the output distribution, not just the inputs.** Booking.com's engineers formalised this as
  *Response Distribution Analysis* in [150 Successful Machine Learning
  Models](https://dl.acm.org/doi/10.1145/3292500.3330744) (KDD 2019) — read the histogram of the
  model's own scores for pathologies, on the reasoning that a healthy binary classifier's scores are
  bimodal near 0 and 1, so a sudden smooth hump in the middle is a symptom even with no label in
  sight. The [MLOps](https://howaiworks.ai/glossary/mlops) page covers the retraining lifecycle this feeds.
- **Instrument the feature pipeline itself**: null rate, out-of-range rate, cardinality, freshness
  per feature. Most "model degradation" incidents are really upstream data incidents, and these
  catch them in minutes rather than months.
- **Find a proxy that settles fast.** For a recommender, click-through today stands in for
  conversion next week; for a churn model, a support-ticket rate moves before the cancellation does.
  A proxy is not the objective, and treating it as one is its own failure — but a noisy signal today
  beats a clean signal in October.

## Real-World Applications

**Google Flu Trends is the canonical published case of a model that decayed in plain sight.** It
estimated influenza-like illness from search volume, and it worked well enough to be held up as the
exemplar of big data. Then, as Lazer, Kennedy, King and Vespignani document in
[The Parable of Google Flu](https://gking.harvard.edu/files/gking/files/0314policyforumff.pdf)
(*Science*, 14 March 2014), *Nature* reported in February 2013 that GFT was "predicting more than
double the proportion of doctor visits for influenza-like illness (ILI) than the Centers for Disease
Control and Prevention". This was not a spike. The paper shows GFT "missed high for 100 out of 108
weeks starting with August 2011", and that by then "even 3-week-old CDC data do a better job of
projecting current flu prevalence than GFT".

The monitoring lessons are unusually clean. First, the cause was drift in a dependency nobody
controlled: the authors note that "the official Google search blog reported 86 changes in June and
July 2012 alone", so the search behaviour generating the features was being reshaped continuously by
a system with its own commercial goals. Second — and this is the part worth internalising — **the
failure was only detectable because the CDC published independent ground truth on a two-week lag.**
The slow, inconvenient, old-fashioned label stream is the only reason anyone found out. Teams that
switch off their expensive label collection because "the model is working" are switching off the
instrument that would tell them otherwise.

**The Epic Sepsis Model shows what happens when nobody measures at all.** ESM is a proprietary
early-warning model "implemented at hundreds of US hospitals". In
[External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized
Patients](https://pmc.ncbi.nlm.nih.gov/articles/PMC8218233/) (*JAMA Internal Medicine*, August 2021),
Wong and colleagues scored it against 38,455 hospitalisations at Michigan Medicine, of which 2,552
(7%) met the sepsis definition. The model's area under the ROC curve was **0.63 (95% CI 0.62–0.64)**
— against the "AUC, 0.76-0.83" that the paper says Epic Systems reported in internal documentation.
It missed **1,709 of the 2,552 sepsis cases (67%)** while alerting on **6,971 of all 38,455
hospitalisations (18%)**.

Do the division the paper implies: 2,552 − 1,709 = 843 sepsis cases detected, from 6,971 alerts.
That is a precision of **12%** — roughly seven alerts without sepsis for every alert with it, which
is exactly the positive predictive value and the corrected number-needed-to-evaluate of 8 that the
paper's Table 2 reports. The model had been running in hospitals for years. It took an academic
health system with its own labelled outcomes to find out. A vendor's reported AUC is a claim about
the vendor's population; it is not a monitor.

**What teams actually install** falls along the same seam as the two layers above. Prometheus and
Grafana, or a cloud equivalent such as CloudWatch, cover the serving layer. Drift and data-quality
reports come from Evidently, WhyLabs, Arize or Fiddler, or from the managed services built into
SageMaker and Vertex AI. For LLM systems the unit of observation is a trace rather than a metric, and
LangSmith and Langfuse record prompt, response, token counts and latency per call. The split is not
cosmetic: the first tells you the service is up, the second tells you the inputs still look like the
training set, and the third is the only place a wrong-but-fluent answer leaves a record.

## Key Concepts

- **Reference window**: the "normal" everything is compared against. Comparing to the training set
  catches everything but alarms forever after the first legitimate shift; comparing to last week
  catches sudden breaks and is structurally blind to slow ones. This choice, not the threshold,
  decides which failures you can see.
- **Label delay**: the lag between a prediction and its ground truth. It ranges from seconds (ad
  click) to a quarter (chargeback, churn) to never (the loan you declined has no repayment history).
  It sets an upper bound on how fast any accuracy-based alarm can possibly fire.
- **Proxy metric**: a fast-settling stand-in for the real objective, used because the real objective
  is unobservable today. Useful precisely to the extent that you remember it is a proxy.
- **Alert precision**: of everything that fires, the share that corresponds to something worth
  doing. You can estimate it in advance from your test count and false-positive rate, and it
  predicts whether anyone will still be reading the channel in six months.
- **Segment monitoring**: aggregate metrics conceal subgroup collapse. Overall accuracy can hold
  steady at 91% while performance on a segment that is 4% of traffic falls off a cliff — and that
  segment is usually where the reputational and regulatory risk lives.

## Challenges

**Alert fatigue is arithmetic, not a personality flaw.** Suppose you run a drift test on 40 features
once a day, each tuned to a 1% false-positive rate. The expected number of false alarms per day is
`40 × 0.01 = 0.4`, and the probability that at least one fires is `1 − 0.99⁴⁰ = 33%`. A third of all
days carry a spurious alert. Over a quarter that is about **36 false alarms**; if genuinely
actionable drift occurs, say, once a quarter, alert precision is `1/37 = 2.7%`. Muting the channel is
the rational response to a channel that is 97% noise. Fixing it means correcting for the number of
tests — one false alarm a month across 40 daily checks needs a per-test rate of
`1/(40 × 30) = 0.00083` — and then accepting that you have traded away sensitivity to small real
shifts, which is a decision to make deliberately rather than discover later.

**The reference window has no correct answer.** Take a feature drifting linearly, the bottom of four
bins shedding 0.3 percentage points of mass a week and the top gaining the same. Week-over-week the
PSI is **0.000125** — nothing, forever. After 52 weeks the distribution has gone from 25/25/25/25 to
9/20/30/41 and the PSI against the original reference is **0.250**, right on the "action required"
line. Same feature, same drift, two windows: one check would never have fired, the other fires
exactly once. Seasonality makes it worse, because a retail model compared against last month alarms
every December for reasons that are not a defect.

**The loud failures and the quiet ones need different instruments.** A feature pipeline that starts
emitting nulls, which the serving code helpfully imputes as `0.0`, is loud: if `days_since_login`
collapses from ten populated bins into a single zero bin, PSI (with a 10⁻⁶ floor on empty bins) comes
out near **12.4**, some fifty times the 0.25 threshold. Input monitoring catches that within a cycle,
and the model meanwhile is confidently telling you every user logged in today. The quiet failure is
concept drift, which produces no input signal at all and shows up only when the labels land. Building
only the first kind of monitor and believing you are covered is the commonest mistake in this area.

**Monitoring is not free, and its cost scales with traffic rather than with model size.** Logging 40
float64 features per request is 320 bytes; at 1,000 requests per second that is 86.4 million rows and
**27.6 GB a day**, about **10 TB a year** before indexes or replication. Teams respond by sampling —
1% brings it to 276 MB a day — which is correct until you need to debug a rare segment that sampling
threw away, or until your 200-row windows put you back at the noise floor calculated above. There is
no configuration that is cheap, complete and statistically sound at once.

**Generative systems have no accuracy metric to degrade.** For a classifier, monitoring eventually
gets a number. For a system that writes text, there is no label arriving in October either; quality
is scored by a rubric, by humans, or by another model acting as judge. A judge is itself a
non-deterministic component whose scores move run to run, so a quality regression has to be
distinguished from judge variance, which needs enough samples to be a statistics problem rather than
a glance at a dashboard. Meanwhile the failure that matters — a fluent, confident, wrong answer —
produces a perfectly normal 200 response of perfectly normal length.

## Future Trends

**Monitoring is moving from the weights to the provider.** When the model is a hosted
[large language model](https://howaiworks.ai/glossary/large-language-model) endpoint, there is nothing on your side to
drift: no training set, no weights, no feature pipeline you own. What can change is the thing behind
the URL. Google Flu Trends failed partly because Google changed its own search product 86 times in
two months while GFT's features depended on it — the same structure, twenty years earlier. So the
monitored objects become the model identifier (pin a dated snapshot, not a floating alias), the
provider's deprecation schedule, and a fixed evaluation set replayed on a cadence with its scores
stored as a time series. That last one is the closest thing to a drift detector available when the
weights are not yours: if the eval score moves and your prompt did not, the endpoint did.

Three other shifts are already visible in what teams build:

- **Drift detection is moving into embedding space.** PSI over token frequencies is close to
  meaningless for free text. The workable substitute is to embed each request and monitor the
  distribution of distances to the reference set's centroids — the same statistic, applied to a
  learned representation rather than a raw feature. See [embedding](https://howaiworks.ai/glossary/embedding).
- **The judge becomes a monitored dependency.** If an LLM scores your outputs nightly, its own
  version, prompt and temperature belong in the change log next to the model's, because a quality
  "regression" that coincides with a judge upgrade is a measurement artefact.
- **Cost per request joins correctness as a first-class metric.** A prompt template edit that adds
  400 tokens of instructions raises the bill on every call forever, breaks nothing, fails no test,
  and appears in no accuracy metric. It is a monitoring problem that did not exist when the model was
  a binary on your own hardware.

## Code Example

The block below does three things: one PSI by hand on four bins, a check that PSI really is the
symmetrised KL divergence, and the part that changes practice — a simulation of what PSI reads when
two windows are drawn from the *same* distribution, against the chi-square prediction.

```python
import numpy as np

rng = np.random.default_rng(0)

def psi(reference, window):
    """Population Stability Index between two binned distributions, in nats."""
    return float(np.sum((window - reference) * np.log(window / reference)))

# Four bins of "transaction amount", cut at the training-set quartiles, so the
# reference is 25% per bin by construction. Two later days, same four bins.
reference = np.array([0.25, 0.25, 0.25, 0.25])
tuesday   = np.array([0.18, 0.22, 0.28, 0.32])
friday    = np.array([0.10, 0.15, 0.30, 0.45])

print(f"Tuesday PSI = {psi(reference, tuesday):.4f}")
print(f"Friday  PSI = {psi(reference, friday):.4f}")

# PSI is exactly KL(a||b) + KL(b||a) — a symmetrised Kullback-Leibler divergence.
kl = lambda p, q: float(np.sum(p * np.log(p / q)))
print(f"KL(friday||ref) {kl(friday, reference):.4f} + "
      f"KL(ref||friday) {kl(reference, friday):.4f} = "
      f"{kl(friday, reference) + kl(reference, friday):.4f}")

# Now the question the thresholds never answer: what does PSI read when
# NOTHING has changed? Two windows of n rows and B = 10 bins, both drawn from
# the same distribution. Yurdakul & Naranjo: (1/n + 1/m)^-1 * PSI is
# approximately chi-square with B - 1 degrees of freedom.
CHI2_99_9DF = 21.666
print()
for n in (200, 1_000, 10_000):
    p = [0.1] * 10
    noise = [psi(rng.multinomial(n, p) / n, rng.multinomial(n, p) / n)
             for _ in range(4_000)]
    print(f"n={n:>6}  simulated mean {np.mean(noise):.4f}  99th pct "
          f"{np.quantile(noise, 0.99):.4f}   predicted mean {2 * 9 / n:.4f}  "
          f"99th pct {CHI2_99_9DF * 2 / n:.4f}")
```

Output:

```
Tuesday PSI = 0.0475
Friday  PSI = 0.3152
KL(friday||ref) 0.1509 + KL(ref||friday) 0.1643 = 0.3152

n=   200  simulated mean 0.0917  99th pct 0.2229   predicted mean 0.0900  99th pct 0.2167
n=  1000  simulated mean 0.0182  99th pct 0.0434   predicted mean 0.0180  99th pct 0.0433
n= 10000  simulated mean 0.0018  99th pct 0.0042   predicted mean 0.0018  99th pct 0.0043
```

The theory and the simulation agree to three digits, and the last block is the practical payload. At
200 rows per window the average PSI between two identical distributions is 0.092 and one day in a
hundred exceeds 0.223 — so the standard 0.10 and 0.25 thresholds fire constantly on nothing. At
10,000 rows the same thresholds are so far above the noise that a real 0.05 shift would never be
reported. **A drift threshold that is not a function of your window size is not a threshold, it is a
superstition**, and the fix is one line: set the alert at `CHI2_99 × (1/n + 1/m)` for the bin count
you actually use, and you know your own false-alarm rate.

## Frequently Asked Questions

### Why does a model need monitoring if the code has not changed?

Because a model's correctness depends on an assumption that nothing in your repository controls: that tomorrow's data resembles the data it was trained on. The code is deterministic; the world is not. A model can serve every request successfully, at normal latency, with a zero error rate, while being wrong far more often than it was last quarter.

### What is the difference between data drift and concept drift?

Data drift is a change in the inputs — the distribution of the features arriving at the model. Concept drift is a change in the relationship between inputs and the correct answer, so the same input now deserves a different prediction. The practical difference is that data drift is visible today from the requests alone, and concept drift is invisible until labelled outcomes arrive.

### How do you monitor accuracy when the true labels arrive months later?

You do not — not in real time. You monitor the things that are observable without labels: the distribution of each input feature, the distribution of the model's own output scores, the rate of nulls and out-of-range values in the feature pipeline, and business proxies that settle faster than the label. Accuracy is then backfilled later, so that a number computed in October describes July.

### What PSI value should trigger an alert?

The industry rule of thumb is 0.10 for 'moderate change' and 0.25 for 'significant change, action required', but Yurdakul and Naranjo note those benchmarks are used with no reference to type I or type II error rates. The honest threshold depends on your window size: with ten bins and only 200 rows per window, two identical distributions already produce an average PSI near 0.09.

### Why do drift alerts get ignored?

Arithmetic. Testing 40 features once a day at a 1% false-positive rate produces 0.4 false alarms a day, so about a third of days carry at least one. Over a quarter that is roughly 36 false alarms against maybe one genuine shift — under 3% of alerts are real, and a team that mutes the channel is responding rationally.

### What do you monitor when the model is someone else's hosted API?

There are no weights to watch, so you watch the provider and your own outputs: pin a dated model snapshot rather than a floating alias, run a fixed evaluation set on a schedule and store the scores as a time series, track tokens and cost per request, and log the full prompt and response so a behaviour change can be diagnosed after the fact.

## Related

### Related terms

- [Machine Learning Operations (MLOps)](https://howaiworks.ai/glossary/mlops)
- [Model Deployment](https://howaiworks.ai/glossary/model-deployment)
- [Production Systems](https://howaiworks.ai/glossary/production-systems)
- [Anomaly Detection (AD)](https://howaiworks.ai/glossary/anomaly-detection)
- [Inference](https://howaiworks.ai/glossary/inference)
- [Benchmark](https://howaiworks.ai/glossary/benchmark)

---

Source: https://howaiworks.ai/glossary/monitoring — HowAIWorks.ai