Term tag: ensemble learning

Ensemble Methods

Why averaging models reduces error and where it stops: the variance of a mean of m correlated predictors, the correlation floor, and what bagging cannot fix.

Gradient Boosting

Each tree is fitted to the negative gradient of the loss, not to the residuals — a boosting round worked by hand, and what changes under log loss.

Low Variance

Low variance means values sit close to their average — and for a model, that retraining it on fresh data barely moves its predictions. Worked on numbers.

Random Forest (RF)

A random forest averages many decision trees, each grown on a bootstrap sample and split on a random subset of features, to cut variance without adding bias.