Definition
Class imbalance is when the categories a classifier is trained to tell apart appear in very unequal numbers — one class is common and another is rare. It matters because of a single uncomfortable fact: on a dataset that is 99% one class, a model that ignores the other class entirely scores 99% accuracy. The headline number looks excellent and the model is worthless, because the thing you actually wanted to find is the thing it never predicts.
This is not a rare edge case. It is the normal shape of the most valuable classification problems. The original SMOTE paper opens by noting that "imbalance on the order of 100 to 1 is prevalent in fraud detection and imbalance of up to 100,000 to 1 has been reported in other applications" (Chawla, Bowyer, Hall & Kegelmeyer, Journal of Artificial Intelligence Research, vol. 16, 2002, pp. 321–357). Fraud, disease, manufacturing defects, network intrusion, rare-event failure — the positive class is rare precisely because it is the expensive, interesting thing, and its rarity is exactly what makes it hard to learn.
The reader who searches this term usually already suspects something is wrong: their model reports a great accuracy but misses every case that counts. The rest of this page explains why that happens mechanically, which metrics see through it, and the four ways to fix it — each with a cost.
How It Works
The 99%-accuracy trap, worked out
Take 10,000 credit-card transactions of which 100 are fraudulent — a 1% base rate, which is mild by fraud standards. Build the laziest possible classifier: it predicts "legitimate" for every transaction and never looks at the features. It is right on all 9,900 legitimate transactions and wrong on all 100 frauds, so its accuracy is 9,900 / 10,000 = 99.00%. It has also caught zero fraud. Every dollar the system was built to save, it lets through.
The reason is structural, not a quirk of this example. Accuracy is a weighted average across classes in which each class counts in proportion to how common it is. When 99% of the rows are legitimate, accuracy is 99% a measurement of the legitimate class and 1% a measurement of fraud. The rare class barely moves the number no matter how badly you handle it, so optimising for accuracy optimises almost entirely for the class you did not care about. This is common enough to have a name — the accuracy paradox — and it is the single most important thing to understand about imbalance.
The fix at the metric level is to use numbers that never count the true negatives. Recall is the share of real frauds the model catches; precision is the share of its alarms that were real. The lazy classifier above has a recall of 0% — the moment you look at recall, the 99% story collapses. The companion trap is the opposite extreme: flag every transaction as fraud and recall jumps to 100%, but precision falls to the base rate of 1%, meaning 99 of every 100 alarms is a false one. The right operating point lives between these poles, and finding it is what the whole field of imbalanced learning is about. The mechanics of precision, recall, F1 and the confusion matrix are worked through in detail on the precision and recall page; this page assumes them and focuses on the imbalance itself.
Why the choice of curve matters
When you summarise a model across all possible thresholds, the imbalance shows up again. A ROC curve plots recall against the false-positive rate, and the false-positive rate has the entire majority class in its denominator — so on 1% positives, thousands of false alarms still look like a small false-positive rate, and the ROC-AUC stays flatteringly high. A precision-recall curve plots precision against recall directly, and precision has the alarm count in its denominator, so it punishes false alarms immediately. The tell is in the random baselines: a no-skill model has an ROC-AUC of 0.5 regardless of imbalance, but its PR-AUC equals the base rate — 0.01 here. On rare-event problems, PR-AUC is the honest summary and ROC-AUC is the reassuring one.
Real-World Applications
Class imbalance is the default condition wherever the event worth predicting is rare and costly.
Credit-card fraud is the canonical case: legitimate transactions outnumber fraudulent ones by roughly 100 to 1 or worse, and the cost of a missed fraud (a chargeback plus investigation) dwarfs the cost of a false alarm (a declined card and an annoyed customer). Fraud teams tune the decision threshold explicitly against those two costs rather than trusting a default, and report recall at a fixed false-alarm budget rather than accuracy.
Disease screening shows the same shape and the same stakes. The SMOTE paper's own running example is a mammography dataset with 10,923 non-cancer cases against 260 cancer cases — about 42 to 1 — and here a false negative is a tumour found a year later while a false positive is a follow-up scan. The metric that matters is recall at a clinically acceptable false-positive rate, never overall accuracy.
Manufacturing defect detection inverts the ratio the other way but keeps the structure: a mature production line may ship well over 99% good units, so a vision system that flags every part "good" scores wonderfully on accuracy and catches no defects. Quality teams therefore track the defect-catch rate (recall) and the re-inspection burden (precision), and accept a higher false-alarm rate than a consumer product would because an escaped defect is expensive.
The common thread: in each domain the two errors have wildly different prices, so the team writes those prices down and picks an operating point, rather than letting a symmetric metric like accuracy pick one silently — and it always picks the wrong one on skewed data.
Key Concepts
Once the metric is honest, you can try to help the model itself. There are four families of remedy, they are routinely combined, and the crucial thing to see is that none of them add information — they only change where the model draws its boundary and how the two errors are traded off.
Resampling: change the training distribution
The bluntest fix is to rebalance the training set. Oversampling duplicates minority examples until the classes are even; undersampling discards majority examples to the same end. Both are cheap and model-agnostic, and both have a matching failure mode. Random oversampling copies the same rare points many times, and a flexible model can memorise those exact points — it draws a tight boundary around them and overfits rather than generalising. Undersampling has the opposite risk: throwing away 98% of the majority class also throws away most of what the model could have learned about it, and on a small dataset that lost information hurts more than the imbalance did.
SMOTE: synthesise instead of copy
SMOTE (Synthetic Minority Over-sampling Technique) was designed to fix oversampling's memorisation
problem by manufacturing new minority examples instead of copying old ones. The mechanism is
simple and worth knowing exactly, because its assumptions are where it breaks. For each minority
sample, SMOTE finds its nearest minority-class neighbors — the original implementation "currently
uses five nearest neighbors" — picks one at random, takes the difference between the two feature
vectors, multiplies that difference by a random number between 0 and 1, and adds it back. In the
paper's pseudocode: gap = random number between 0 and 1, then
Synthetic = Sample[i] + gap * dif. The new point lands somewhere on the straight line between two
real minority examples, filling in the region between them rather than piling copies on top of
them.
That interpolation is also SMOTE's weakness. It assumes the space between two minority neighbors is itself minority territory; where the classes overlap, the line can cross into majority space and manufacture a synthetic "fraud" sitting in a crowd of legitimate points — noise that the model then has to fit around. Because the neighbors are found by Euclidean distance, SMOTE is sensitive to the scale of the features: an unscaled feature measured in the thousands will dominate the distance and distort which points count as neighbors, so feature scaling is effectively a prerequisite. The same distance machinery is why SMOTE is a close cousin of k-nearest-neighbors reasoning, and inherits its troubles in high dimensions.
Class weights: make the rare error cost more
Instead of changing the data, you can change the loss. Class weighting multiplies each example's contribution to the loss by a factor, so that a mistake on a rare example is penalised more heavily than a mistake on a common one. The natural setting is inverse frequency: to make the two classes contribute equally to the loss on our 1%-positive data, weight each minority example by the imbalance ratio — 9,900 / 100 ≈ 99, so a single fraud counts as much as ninety-nine legitimate transactions. This is mathematically close to oversampling by the same ratio but cheaper, because it reweights rather than duplicating rows, and it is the approach the SMOTE authors compare against under the name of varying "class priors in Naive Bayes" or loss ratios in a rule learner. Its cost is that aggressive weighting distorts the model's predicted probabilities, which brings us to the last lever.
Threshold moving: stop deciding at 0.5
A classifier outputs a score; turning that score into a decision needs a threshold, and the default of 0.5 is an arbitrary inheritance from balanced problems. On imbalanced data the score distribution is squashed toward zero, and the threshold that best trades your two costs is usually far below 0.5. Moving it is the cheapest intervention of all — it requires no retraining, only a sweep of the precision-recall curve to find the point that meets your recall target at an acceptable precision. It pairs naturally with the others: resampling and weighting change the shape of the score distribution, threshold moving picks the cut through it. Note that resampling and heavy weighting both leave the raw scores miscalibrated — a score of 0.8 no longer means an 80% chance of fraud — so if you need the probabilities themselves rather than a yes/no decision, you will have to recalibrate after rebalancing, or the threshold you pick will not mean what you think.
Challenges
The mistakes that actually sink imbalanced projects are rarely about which remedy you pick; they are about how and where you apply it.
Resample the training set only — never the test set. The test set has to mirror the real world, where the positive class is rare. If you balance it, your reported precision is measured against a 50% base rate that will never occur in production, and every number you quote is inflated. Rebalance inside the training fold and evaluate on the untouched, still-imbalanced distribution.
SMOTE before the split is silent data leakage. If you run SMOTE on the full dataset and then split into train and test, synthetic points interpolated from test-set minority examples end up in the training set. The model has effectively seen the test data, the score jumps a few points, and none of it survives deployment. SMOTE — and every resampling step — belongs strictly inside cross-validation, applied to each training fold after that fold is separated.
A great score on a shifting base rate is a time bomb. The imbalance ratio you tuned against is not a constant of nature. Fraud tactics change, disease prevalence moves with the season, a factory process drifts — and when the base rate shifts, the threshold you carefully chose is now wrong and precision quietly rots. This is concept drift by another name, and it means the operating point is a thing to monitor and re-fit, not a thing to set once.
Do not fix imbalance you do not have. If the classes are already separable enough that the model hits your recall target at some threshold, resampling and reweighting add noise and complexity for nothing, and SMOTE can actively hurt by fabricating minority points in contested regions. The discipline is to plot the precision-recall curve first and intervene only if it shows the model genuinely cannot reach a usable operating point — imbalance is a symptom to diagnose, not a box to tick.
Code Example
The trap is worth seeing in numbers you can re-run. This computes accuracy, recall and precision directly from the confusion-matrix counts for three classifiers on the 1%-fraud dataset above: the lazy "always legitimate" model, the panicked "flag everything" model, and a realistic model that catches 70 of the 100 frauds while raising 130 false alarms.
# One fraud in a hundred: 10,000 transactions, 100 of them fraudulent.
total = 10_000
positives = 100 # fraud (the minority class)
negatives = total - positives
def report(name, tp, fp, fn, tn):
accuracy = (tp + tn) / total
recall = tp / (tp + fn) if (tp + fn) else float("nan")
precision = tp / (tp + fp) if (tp + fp) else float("nan")
print(f"{name:<22} accuracy={accuracy:6.2%} recall={recall:6.2%} "
f"precision={precision if precision != precision else f'{precision:6.2%}'}")
# Model A: always predict "legitimate" — never fires.
report("always-legitimate", tp=0, fp=0, fn=100, tn=negatives)
# Model B: flag every transaction — catches all fraud, drowns in alarms.
report("flag-everything", tp=100, fp=negatives, fn=0, tn=0)
# Model C: a real classifier — catches 70 frauds, raises 130 false alarms.
report("real-classifier", tp=70, fp=130, fn=30, tn=negatives - 130)
Running it prints:
always-legitimate accuracy=99.00% recall= 0.00% precision=nan
flag-everything accuracy= 1.00% recall=100.00% precision= 1.00%
real-classifier accuracy=98.40% recall=70.00% precision=35.00%
Read the three rows together and the point is unmissable. The do-nothing model has the second-best accuracy in the table and is the only one that is completely useless. The flag-everything model has by far the worst accuracy — 1% — yet it catches every fraud, because accuracy on skewed data can be actively anti-correlated with doing the job. The real classifier, which is the only sane choice, scores lower accuracy than the model that does nothing. Any process that ranks these three by accuracy would pick exactly the wrong one, which is the whole reason class imbalance has a page.