Term tag: generalization

Bias-Variance Tradeoff

Statistical bias, not fairness. Error splits into bias squared, variance and irreducible noise — derived, worked on numbers, and honest about where it breaks.

Cross-Validation (CV)

How cross-validation turns one lucky train/test split into a stable estimate, why 5 or 10 folds, and the leakage that quietly inflates every score.

Data Augmentation

Making new training examples by transforming existing ones. Valid only when the label is indifferent to the transform — and it adds variety, not information.

Early Stopping

Early stopping halts training once validation loss stops improving, so the model generalises instead of memorising. How patience and best-weight restore work.

Generalization

Why a model that scores 99% in training fails in the world: the generalization gap, the test-set size needed to measure it, and why classical theory is wrong.

Machine Learning (ML)

How a machine finds a rule nobody wrote: a line fitted to four points by hand, loss falling from 433.5 to 6.38 in three steps, and why 98% accuracy lies.

Overfitting

Overfitting is when a model memorizes its training data, including the noise, so it scores high on training data but fails on new, unseen data.

Regularization

Regularization deliberately constrains a model so it fits training data less perfectly, trading a little training accuracy for much better generalization.

Underfitting

Underfitting is when a model is too simple to capture the real pattern, so it does poorly on both training and new data — the mirror of overfitting.