Term tag: boosting

Ensemble Methods

Why averaging models reduces error and where it stops: the variance of a mean of m correlated predictors, the correlation floor, and what bagging cannot fix.

Gradient Boosting

Each tree is fitted to the negative gradient of the loss, not to the residuals — a boosting round worked by hand, and what changes under log loss.