Underfitting

Definition

Underfitting is when a machine learning model is too simple to capture the real pattern in its training data, so it does poorly on the data it trained on and poorly on new, unseen data. It is the mirror image of overfitting: an overfit model memorizes the training data and scores high there while failing on fresh data; an underfit model never manages to learn the training data in the first place, so it fails on both. If overfitting is answering the exam you already saw but not the new one, underfitting is failing both exams the same way.

The reason underfitting is easy to miss is that the number people watch to catch overfitting — the gap between training error and test error — stays small when a model underfits. Both numbers are bad together. The signature of underfitting is therefore the training error itself: an overfit model drives training error toward zero, while an underfit model cannot get it down at all. A model that scores 60% on training data and 59% on new data has an almost invisible gap and looks "well-generalized," but it is not — it is uniformly weak. It has generalized its own failure.

How It Works

A model has some amount of capacity — how many distinct shapes it can represent. A straight line has very little; a deep neural network with millions of weights has an enormous amount. Underfitting is the low-capacity end of that scale: the model does not have enough freedom — or enough training, or the right inputs — to represent the shape of the true relationship, so the best fit it can find still leaves large, systematic errors. It is wrong in the same direction across whole regions of the input, not just noisily off by a little.

The cleanest way to see this is the same polynomial-fitting setup used to demonstrate overfitting, run from the underfit end. Fit a straight line to data that clearly curves and the line simply cannot bend to follow it: no matter how you tune its two coefficients, it rides above the curve in some regions and below it in others. The ## Code Example below fits a straight line (degree 1) to 40 low-noise points drawn from a curved signal. Its training error settles at an RMSE of 0.714 while the measurement noise built into the data is only 0.1 — the model's own residuals are about seven times the noise, which tells you the error is coming from the model, not the data. A degree-7 polynomial, flexible enough to follow the curve, brings training RMSE down to 0.067, right at the noise floor. The straight line had the capacity to do neither.

This is what makes underfitting the easier of the two failures to detect. To catch overfitting you need a held-out validation set, because training performance looks perfect and only fresh data exposes the problem. To catch underfitting you barely need one: a model that cannot fit the data it was allowed to see will not do better on data it was not. In the code table the straight line scores 0.714 on training and 0.693 on the 200 held-out points — a gap of about 3%, both numbers bad and nearly equal. An overfit degree-11 fit of the same signal does the reverse: training error 0.000, test error exploding past 3. Watching only the gap would flag the overfit model and wave the underfit one through as the "safe," well-behaved choice.

In the language of the bias-variance tradeoff, underfitting is high bias and low variance. The model is rigid, so it makes roughly the same wrong predictions no matter which sample of data you train it on — its errors are stable and systematic rather than jumping around from one training run to the next. Overfitting sits in the opposite corner, low bias and high variance, its fit swinging wildly with every reshuffle of the data. The two are ends of one dial, and every fix for underfitting spends some variance to buy the bias down — the exact reverse of what regularization does when it fixes overfitting.

Real-World Applications

Underfitting rarely announces itself with a dramatic failure; it reaches production disguised as a modest, stable-looking result. The most consequential place it hides is the deliberate choice of a simple model. In regulated domains — credit scoring, insurance pricing, some clinical risk scores — teams often must use a linear or logistic model, or a small point-based scorecard, because the decision has to be auditable and explainable to a regulator. That is a legitimate trade, but it accepts a real amount of underfitting: a linear model applied to a genuinely nonlinear relationship leaves signal on the table by construction, and the cost is a permanent ceiling on accuracy that no amount of tuning within the model class removes. The engineering question is not "did we pick the simplest model" but "is the accuracy we gave up for interpretability a price this decision can afford."

The other common appearance is the baseline that quietly ships. A sensible workflow starts with a simple model — a linear regression, a logistic baseline — to establish a floor before anything fancier is justified. When that baseline plateaus at mediocre accuracy on both the training and the validation set, the correct reading is "this model class is too weak, add capacity," and the correct next move is a more expressive model or better features. The failure mode is to instead read the small train/validation gap as a sign of health, declare the baseline "robust and well-generalized," and stop. The model is not robust; it is under-powered, and the gap was small only because the model was equally weak everywhere. Distinguishing those two readings is the practical skill underfitting demands.

Key Concepts

Fixing underfitting means giving the model more room, or better material, to fit the data — the opposite of the constraints you add to fight overfitting. Roughly in the order worth trying:

• More capacity. Move to a more expressive model — more features, more parameters, more layers, or a nonlinear model in place of a linear one. This directly attacks the cause: the model could not represent the true shape, so you enlarge the set of shapes it can represent. Overshoot and you cross into overfitting, which is why this is a dial, not a switch.
• Train longer, or optimize better. Sometimes the capacity is there but the model never reached it — training error was still falling when you stopped. More epochs, a higher or scheduled learning rate, or a better optimizer lets it finish converging. The tell is a training-loss curve that had not yet flattened.
• Reduce over-aggressive regularization. Regularization — L2 weight decay, dropout — deliberately forces simplicity to prevent overfitting. Set too strong, it forces more simplicity than the data warrants and manufactures underfitting. If both errors are high, turning it down is a cheap first experiment.
• Better features. Even a high-capacity model underfits if the signal is not in its inputs. Predicting house prices without a location feature, or fraud without transaction velocity, leaves the model reaching for a pattern it was never given the raw material to find. Feature engineering adds that material.

Challenges

The hard part of underfitting is that its most common cause of harm is a misread of a healthy-looking number. A small gap between training and validation error is treated as the goal — it is what you chase when preventing overfitting — so a model whose gap is small because both numbers are bad gets mistaken for a well-generalized one and shipped. The guard is simple to state and easy to forget: read the level of the training error, not just the gap. A 59% validation score next to a 60% training score is not a well-generalized model; it is a uniformly weak one, and the near-zero gap is telling you nothing except that the two failures agree.

The subtler challenge is telling underfitting apart from a genuinely hard problem. If the data carries a lot of irreducible noise, or the inputs simply do not determine the output, then low accuracy is the ceiling, not a fixable flaw — and piling on capacity to chase it just walks you past a good fit into overfitting, buying variance for no reduction in real error. The way to tell them apart is to compare training error against an estimate of the noise floor rather than against a perfect score: in the code example, a training RMSE of 0.714 against noise of 0.1 is clearly the model underfitting, whereas a training RMSE already down near 0.1 would mean the model has learned everything there is to learn and the remaining error is the data's, not the model's. Get that comparison wrong in either direction and you either ship a model that is too weak or over-engineer one chasing noise it can never remove.

Code Example

This fits polynomials of three complexities to the same 40 low-noise points from a curved signal, and reports error on the training points versus 200 unseen points from the true curve. Run it as-is to reproduce the numbers cited above.

import numpy as np

rng = np.random.default_rng(0)

# True signal: a clearly curved relationship. Noise is small, so a model with
# enough capacity can get near zero error — which makes underfitting stand out.
f = lambda x: np.sin(1.5 * x)

# 40 training points with light measurement noise; 200 clean test points.
x_train = np.linspace(-3, 3, 40)
y_train = f(x_train) + rng.normal(0, 0.1, size=x_train.shape)
x_test = np.linspace(-3, 3, 200)
y_test = f(x_test)

def rmse(a, b):
    return float(np.sqrt(np.mean((a - b) ** 2)))

print(f"{'degree':>6} {'train RMSE':>11} {'test RMSE':>10}")
for degree in (1, 3, 7):
    coeffs = np.polyfit(x_train, y_train, degree)   # least-squares fit
    tr = rmse(np.polyval(coeffs, x_train), y_train)
    te = rmse(np.polyval(coeffs, x_test), y_test)
    print(f"{degree:>6} {tr:>11.3f} {te:>10.3f}")

Output:

degree  train RMSE  test RMSE
     1       0.714      0.693
     3       0.315      0.270
     7       0.067      0.042

Read the rows from the top as underfit toward well-fit. Degree 1 (a straight line) is too rigid to follow the curve, so it scores 0.714 on training and 0.693 on unseen data — both far above the 0.1 noise floor, and only 3% apart. That small gap is the trap: the two numbers agree, but they agree on being bad. Degree 3 already halves the error on both. Degree 7 has enough capacity to trace the real shape, landing at 0.067 and 0.042 — near the noise floor, low on both, a genuinely good fit. Underfitting is the top row: not a gap to close, but a training error that never came down.