Term tag: regularization

Bias-Variance Tradeoff

Statistical bias, not fairness. Error splits into bias squared, variance and irreducible noise — derived, worked on numbers, and honest about where it breaks.

Early Stopping

Early stopping halts training once validation loss stops improving, so the model generalises instead of memorising. How patience and best-weight restore work.

Feature Selection (FS)

Which columns to keep and how to decide: filter vs wrapper vs embedded, why 2^n subsets rules out brute force, and the CV mistake that fakes accuracy.

Hyperparameter

A setting you choose before training that gradient descent never updates — learning rate, batch size, layer count — and why a bad one caps the model.

Membership Inference

A privacy attack that decides whether a specific record was in a model's training set, exploiting that models are more confident on data they memorized.

Overfitting

Overfitting is when a model memorizes its training data, including the noise, so it scores high on training data but fails on new, unseen data.

Regularization

Regularization deliberately constrains a model so it fits training data less perfectly, trading a little training accuracy for much better generalization.

Underfitting

Underfitting is when a model is too simple to capture the real pattern, so it does poorly on both training and new data — the mirror of overfitting.