Term tag: Fine-tuning

Direct Preference Optimization (DPO)

DPO trains a model on human preferences without a reward model or reinforcement learning: the derivation, the memory it saves, and where it loses to PPO.

Fine-tuning (FT)

What fine-tuning changes inside a model, what it costs in GPU memory, and why it reliably buys format and style but not new facts.

Low-Rank Adaptation (LoRA)

LoRA freezes a model's weights and trains two thin matrices beside each one: 98,304 numbers instead of 151 million, and no added inference latency.

Pre-trained Models

A neural network someone else already trained on a huge general dataset, which you download and adapt to your task instead of training from scratch.

Simple Continual Pretraining (SCP)

Method for turning a pretrained autoregressive LLM into a diffusion language model by continuing pretraining with a bidirectional attention mask.

Supervised Fine-Tuning (SFT)

Supervised fine-tuning trains a pretrained language model on curated prompt-response pairs with next-token loss, teaching it to follow instructions.

Transfer Learning (TL)

Transfer learning reuses a model trained on one large task as the starting point for a related task, so you need far less labeled data and compute.