Term tag: training

Active Learning

Active learning lets a model choose which unlabelled examples to get labelled next, hitting target accuracy with far fewer labels than random labelling.

Backpropagation

Backpropagation computes the gradient of the loss for every parameter at once, for about the cost of one extra forward pass. How the chain rule does it.

Deep Learning

What makes deep learning deep: each layer builds on the one below, so capacity grows multiplicatively with depth and only additively with width.

Early Stopping

Early stopping halts training once validation loss stops improving, so the model generalises instead of memorising. How patience and best-weight restore work.

Gradient Descent

The rule that turns a gradient into a weight update. Worked on f(x)=x²: why the step size alone decides between converging, oscillating and blowing up.

Hyperparameter

A setting you choose before training that gradient descent never updates — learning rate, batch size, layer count — and why a bad one caps the model.

Loss Function

The number a model is trained to make smaller. Its slope, not its value, moves every weight — so the loss you pick decides what the model learns.

Machine Learning (ML)

How a machine finds a rule nobody wrote: a line fitted to four points by hand, loss falling from 433.5 to 6.38 in three steps, and why 98% accuracy lies.

Neural Network

Why a stack of layers does what one cannot: two linear layers collapse into a single matrix, and a hidden layer of two units solves XOR.

Normalization (LayerNorm, RMSNorm, BatchNorm)

Normalization rescales a layer's activations to a stable mean and scale so deep networks train faster. How BatchNorm, LayerNorm and RMSNorm differ.

Optimization

In machine learning, optimization means minimizing a loss function to fit a model's parameters — the iterative, gradient-driven mechanism behind training.

Training

Training fits a model's parameters to data: forward pass, measure the loss, backward pass, nudge every weight downhill, repeat until the loss stops falling.

Weights

A weight is a single learned number setting the strength of a connection between two neurons; together, the weights are the trained model.