Term tag: decision trees

Decision Trees (DT)

How a decision tree actually chooses each split: Gini impurity worked by hand, why depth drives overfitting, and what a single tree cannot represent.

Gradient Boosting

Each tree is fitted to the negative gradient of the loss, not to the residuals — a boosting round worked by hand, and what changes under log loss.

Information Gain

Entropy worked from scratch, one split computed by hand in bits, and why a unique ID column scores the maximum possible information gain.