Term tag: tree-based models

Decision Trees (DT)

How a decision tree actually chooses each split: Gini impurity worked by hand, why depth drives overfitting, and what a single tree cannot represent.

Ensemble Methods

Why averaging models reduces error and where it stops: the variance of a mean of m correlated predictors, the correlation floor, and what bagging cannot fix.

Gradient Boosting

Each tree is fitted to the negative gradient of the loss, not to the residuals — a boosting round worked by hand, and what changes under log loss.

Random Forest (RF)

A random forest averages many decision trees, each grown on a bootstrap sample and split on a random subset of features, to cut variance without adding bias.

Support Vector Machines (SVM)

A classifier that separates two classes with the widest possible margin, using support vectors and the kernel trick to handle non-linear data.