Term tag: gradient descent

Backpropagation

Backpropagation computes the gradient of the loss for every parameter at once, for about the cost of one extra forward pass. How the chain rule does it.

Early Stopping

Early stopping halts training once validation loss stops improving, so the model generalises instead of memorising. How patience and best-weight restore work.

Feature Scaling

Feature scaling puts numeric features on comparable ranges so no variable dominates by its units alone — why distance- and gradient-based models need it.

Gradient Descent

The rule that turns a gradient into a weight update. Worked on f(x)=x²: why the step size alone decides between converging, oscillating and blowing up.

Hyperparameter

A setting you choose before training that gradient descent never updates — learning rate, batch size, layer count — and why a bad one caps the model.

Machine Learning (ML)

How a machine finds a rule nobody wrote: a line fitted to four points by hand, loss falling from 433.5 to 6.38 in three steps, and why 98% accuracy lies.

Neural Network

Why a stack of layers does what one cannot: two linear layers collapse into a single matrix, and a hidden layer of two units solves XOR.

Optimization

In machine learning, optimization means minimizing a loss function to fit a model's parameters — the iterative, gradient-driven mechanism behind training.

Training

Training fits a model's parameters to data: forward pass, measure the loss, backward pass, nudge every weight downhill, repeat until the loss stops falling.