Definition
Transfer learning is reusing a model that was already trained on one large, general task as the starting point for a different but related task — you keep the general-purpose features it already learned and retrain only the parts that are specific to your problem. Because most of the model is inherited rather than learned from scratch, you can reach strong accuracy with far less labeled data and far less compute than training a fresh model would take.
A concrete version of the idea: an image model trained on ImageNet has seen roughly 1.28 million labeled images across 1,000 categories, and along the way it learned to detect edges, textures and shapes that are useful for almost any vision task. If you want a classifier that tells apart ten kinds of skin lesion, you do not have a million labeled examples and you do not need them. You take that pre-trained model, keep the general visual machinery, and train only a small new classifier on your few thousand images.
Three terms sit close together and are worth separating up front. A pre-trained model is the thing you transfer from — the reusable network that already did the big general training. Transfer learning is the general principle of reusing it on a new task. Fine-tuning is one specific technique under that principle, where you continue training some of the pre-trained weights on your data. This page is about the principle; the individual techniques link out to their own pages.
How It Works
Transfer learning works because of a fact about how deep networks organize what they learn: the layers form a gradient from general to specific. The early layers learn broad, reusable features — in vision, edges, colors and textures; in language, basic word and character patterns — and these are useful for essentially any task in that domain. The late layers combine those features into something that answers the original training task specifically: "is this ImageNet class 273 or 274?" That answer is exactly the part you want to throw away.
The mechanism follows directly. You freeze the early layers so their weights do not change, and you retrain or replace the late layers with a new "head" aimed at your task. Freezing means the general features you inherited are protected while your small amount of task-specific training happens on top of them. The one dial that matters is how many layers you freeze, and it defines the two standard strategies.
Feature extraction freezes the entire pre-trained network and trains only a new classifier on top of its output features. The pre-trained model becomes a fixed function that turns an input into a rich feature vector, and you learn a simple mapping from that vector to your labels. This is fast, needs the least data, and cannot damage the pre-trained features because it never touches them.
Fine-tuning goes further: you unfreeze some of the later layers and continue training them on your data, usually at a low learning rate so the inherited weights are nudged rather than overwritten. This lets the model specialize its higher-level features to your task, which helps when your task is far enough from the original that the frozen features are not quite right — at the cost of needing more data and more care to avoid destroying what was already learned.
Here is the split on real numbers. A ResNet-50 has 50 weighted layers, about 25.56 million parameters, and produces a 2,048-dimensional feature vector just before its final classifier. To adapt it to a 10-class problem by feature extraction, you freeze 49 of the 50 layers and train a single new layer that maps 2,048 features to 10 classes. That layer has 2048 × 10 + 10 = 20,490 parameters — roughly 0.09% of the network. You are learning twenty thousand numbers instead of twenty-five million, which is why a few thousand labeled examples suffice. Fine-tuning the same model might instead freeze the first ~40 layers and unfreeze the last stage plus the head, training on the order of ten of the fifty layers.
The choice of which layers to transfer is not arbitrary, and the foundational study of it — Yosinski and colleagues' 2014 paper "How transferable are features in deep neural networks?" — measured it directly on an 8-layer ImageNet network by transferring the first n layers for every n from 1 to 7 and retraining the rest. Two findings from that work still shape how practitioners set the freeze point. First, as expected, features grow more task-specific with depth, so transferring more layers eventually transfers the source task's specificity and hurts. Second, and less obviously, neurons in the middle layers co-adapt — they learn to work together in a way that is fragile to being split apart, so cutting a network mid-stack and freezing the bottom half can hurt performance even when those features would have been fine to reuse. The practical rescue is fine-tuning: the paper found that starting from transferred features and then continuing to train them gives a generalization boost that persists even after the model has fully adapted to the target task, because good initialization lands the network in a better region than random weights do.
Types
Two independent distinctions get called "types of transfer learning," and it helps to keep them separate because they answer different questions.
The first is how you adapt the model, and it is the practical axis you tune every time: feature extraction (freeze the whole pre-trained model, train only a new head) versus fine-tuning (unfreeze some pre-trained layers and continue training them at a low learning rate). These are two ends of one continuum — the number of frozen layers — rather than sharply separate methods, and the right point depends on how much target data you have and how close your task is to the original.
The second is what actually differs between the source and the target. When the task itself changes — ImageNet object recognition to detecting tumors — that is inductive transfer, and you need labeled target data to learn the new task. When the task stays the same but the data distribution shifts — a sentiment model trained on product reviews applied to tweets — that is domain adaptation (the transductive case), where the goal is to close the gap between two distributions rather than to learn a new objective. Most everyday transfer learning is inductive; domain adaptation is its own research area with its own methods.
Real-World Applications
Medical imaging is the canonical case for feature extraction and fine-tuning. Hospitals rarely have millions of labeled scans, so systems for reading chest X-rays, retinal photographs and pathology slides routinely start from an ImageNet-pre-trained CNN and fine-tune it on a few thousand to a few tens of thousands of labeled medical images. The widely cited CheXNet chest-X-ray classifier, for example, is a DenseNet initialized from ImageNet weights rather than trained from scratch — the general edge and texture detectors transfer, and only the diagnostic head is learned from clinical data.
Natural language processing was reshaped by exactly this move. BERT is pre-trained on large unlabeled text corpora with self-supervised learning, and downstream teams then fine-tune that single pre-trained model for sentiment analysis, named-entity recognition, or question answering — each with a modest labeled dataset that would never have been enough to train a language model from zero. The pre-training is done once and expensively; the transfer is done many times and cheaply.
Foundation models and LLMs are the largest-scale version of the pattern. A foundation model is pre-trained on internet-scale data and then adapted — by fine-tuning or lighter parameter-efficient methods — to domain-specific tasks like legal document review, customer support, or code generation. The economic logic is transfer learning's: concentrate the enormous training cost in one general model, then let everyone transfer from it.
Challenges
Negative transfer is the failure that defines the method's limits. If the source and target are too dissimilar, the pre-trained features are not just unhelpful but actively harmful, and you end up worse than a model trained from scratch on the same target data. Transferring from natural photographs to, say, radar or spectrogram images can land here: the "edges and textures" the backbone learned are the wrong edges and textures, and they bias the model toward patterns that do not exist in the new domain. There is no guarantee that "related-sounding" tasks are related enough; whether transfer helps is an empirical question you have to check, not an assumption.
Catastrophic forgetting is the danger specific to aggressive fine-tuning. If you unfreeze too many layers or train at too high a learning rate, the gradients from your small target dataset overwrite the general features the model spent enormous compute acquiring — the very thing you transferred for. The model can end up good at your task and no longer good at anything else, and if your dataset is small it may not even be good at your task. The low learning rate and the frozen early layers are the guardrails against this; catastrophic forgetting has its own page on why it happens and how to mitigate it.
Overfitting on tiny target data is the related trap. The whole appeal of transfer learning is using it when you have little data, but the more of a large model you unfreeze, the more capacity you expose to memorize that little data instead of generalizing from it. This is why feature extraction — training only a ~20,000-parameter head — is often the safer starting point on small datasets, and why you unfreeze more layers only as you have more data to justify them. See overfitting for the general phenomenon.
Code Example
Feature extraction in PyTorch. Load a pre-trained ResNet-50, freeze every parameter, then replace only its final classifier with a fresh layer for a 10-class task. Because the frozen parameters have requires_grad = False, the optimizer trains only the new head.
import torch
import torchvision
# Load ResNet-50 with its ImageNet-pretrained weights (the model you transfer FROM).
model = torchvision.models.resnet50(weights="IMAGENET1K_V1")
# Freeze the whole backbone: its general features are inherited, not retrained.
for param in model.parameters():
param.requires_grad = False
# Replace the 2048->1000 ImageNet classifier with a new 2048->10 head.
# A fresh Linear layer defaults to requires_grad = True, so ONLY it will train.
num_classes = 10
model.fc = torch.nn.Linear(in_features=2048, out_features=num_classes)
total = sum(p.numel() for p in model.parameters())
trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)
print(f"total parameters: {total:,}")
print(f"trainable parameters: {trainable:,}")
print(f"training {100 * trainable / total:.4f}% of the model")
Output:
total parameters: 23,528,522
trainable parameters: 20,490
training 0.0871% of the model
To fine-tune instead of extract features, you would unfreeze the last block as well — for example by setting requires_grad = True on the parameters in model.layer4 — and train it together with the head at a low learning rate so the inherited features are adjusted rather than erased.