Term tag: Model Compression

Knowledge Distillation

How a small model learns what a big one knows: the teacher's full probability distribution carries structure the hard labels do not — Hinton's dark knowledge.

Model Compression

Making a trained model smaller and cheaper to run — quantization, pruning, distillation and low-rank methods, and which one your problem actually needs.

Quantization

Running a model in fewer bits — FP16, FP8, INT4 — to cut memory and cost, what accuracy it actually costs, and why a memory-bound model also gets faster.

Small Language Model (SLM)

A language model small enough to run on the device that uses it — roughly under 5B parameters. Trained far past compute-optimal to buy cheap inference.