Term tag: pre-trained models

Fine-tuning (FT)

What fine-tuning changes inside a model, what it costs in GPU memory, and why it reliably buys format and style but not new facts.

Foundation Models

Not a marketing word for a big AI model: a foundation model is trained once on broad data and reused as the base for many downstream tasks.

Pre-trained Models

A neural network someone else already trained on a huge general dataset, which you download and adapt to your task instead of training from scratch.

Self-supervised Learning (SSL)

Training a model on unlabeled data by hiding part of each input and making the model predict it — the paradigm behind next-token prediction and modern LLMs.

Simple Continual Pretraining (SCP)

Method for turning a pretrained autoregressive LLM into a diffusion language model by continuing pretraining with a bidirectional attention mask.

Transfer Learning (TL)

Transfer learning reuses a model trained on one large task as the starting point for a related task, so you need far less labeled data and compute.