Term tag: continual learning

Catastrophic Forgetting

Why fine-tuning makes a model worse at everything it used to do well — the mechanism inside shared weights, and what each fix actually costs.

Simple Continual Pretraining (SCP)

Method for turning a pretrained autoregressive LLM into a diffusion language model by continuing pretraining with a bidirectional attention mask.