Term tag: optimization

Backpropagation

Backpropagation computes the gradient of the loss for every parameter at once, for about the cost of one extra forward pass. How the chain rule does it.

Direct Preference Optimization (DPO)

DPO trains a model on human preferences without a reward model or reinforcement learning: the derivation, the memory it saves, and where it loses to PPO.

Gradient Descent

The rule that turns a gradient into a weight update. Worked on f(x)=x²: why the step size alone decides between converging, oscillating and blowing up.

Group Relative Policy Optimization (GRPO)

GRPO is PPO without the critic: it scores a group of sampled answers against each other. How it works, what it saves, and where it biases training.

Hyperparameter

A setting you choose before training that gradient descent never updates — learning rate, batch size, layer count — and why a bad one caps the model.

Loss Function

The number a model is trained to make smaller. Its slope, not its value, moves every weight — so the loss you pick decides what the model learns.

Optimization

In machine learning, optimization means minimizing a loss function to fit a model's parameters — the iterative, gradient-driven mechanism behind training.