Few-shot Learning (FSL)

Definition

Few-shot learning means getting a model to perform a task you never trained it for by showing it a handful of worked examples — typically three to five, sometimes as few as one. The term now covers two genuinely different things, and this page leads with the one almost everyone arrives looking for: few-shot prompting, where the examples are pasted into the prompt of a large language model alongside the real question.

The surprising part is that few-shot prompting updates no weights at all. The examples are just tokens. They are read, used to condition the answer, and thrown away when the request ends. Nothing about the model is different afterwards; the next request begins with no memory that the examples ever existed. This is why the behaviour is more precisely called in-context learning — the "learning" happens inside a single forward pass and lasts exactly as long as the context does. The older, narrower meaning of few-shot learning is a training regime, covered under Types below, and it does change weights.

Getting this wrong costs money in two directions. Believe the model retained the examples and you will remove them from the prompt and watch quality silently collapse. Believe they are free and you will ship a ten-example prompt on a high-traffic endpoint and pay for those examples on every single call, forever.

How It Works

A few-shot prompt is a single block of text with the demonstrations laid out in the same shape as the real request, followed by the real request with its answer missing:

Review: "Arrived bent, but they refunded me instantly."  Sentiment: mixed
Review: "Cheap plastic, broke in a week."                Sentiment: negative
Review: "Exactly what I needed."                          Sentiment: positive
Review: "Fast delivery, product is fine I guess."         Sentiment:

Nothing special happens to those first three lines. The model is doing what it always does — predicting the next token — and the pattern established above makes positive, negative or mixed overwhelmingly more likely than a paragraph of commentary. That is the whole mechanism. In chat APIs the same thing is expressed as alternating user and assistant turns before the real one, which is why example turns still work on models where other prompt tricks have been removed.

The examples turn out to teach less than they appear to. Min et al. (2022) replaced the labels in the demonstrations with random labels and found performance barely moved, consistently across 12 different models including GPT-3. What the demonstrations mainly supply is the label space (these three words are the permitted answers), the input distribution (inputs look like short product reviews), and the format of the sequence. They are far weaker at teaching the actual input-to-label mapping than the word "learning" implies.

The arithmetic almost nobody does. Take ten demonstrations of 200 tokens each: 2,000 extra input tokens on every request. At Claude Sonnet 5's introductory input rate of $2 per million tokens — in force to 31 August 2026 — that is 2,000 ÷ 1,000,000 × $2 = $0.004 per request, or $4 per thousand requests. Across a million requests you have sent 2 billion tokens of examples and spent $4,000 — to send the same four paragraphs two million times. A one-off fine-tuning run costing $500 pays for itself at roughly 125,000 requests ($500 ÷ $0.004), and afterwards the prompt is 2,000 tokens shorter on every call, which cuts latency too. Notice how sensitive that threshold is to a number you do not control: at the standard $3 rate the same run breaks even near 83,000 requests instead, so a price change of one dollar moves the decision point by a third. That is the real shape of the fine-tuning decision: a recurring cost that scales with traffic, versus a one-off cost that scales with training-set size.

Prompt caching changes the slope but not the shape. If the demonstration block sits at the front of every prompt and is byte-identical each time, providers can cache it and bill reads at roughly a tenth of the normal input rate — turning that $6,000 into something nearer $600. The catch is that caching is a prefix match with a short time-to-live (five minutes by default on Anthropic's API), so one interpolated timestamp ahead of the examples, or a quiet period in traffic, and you are paying full price again.

Types

The two things called few-shot learning share a name and almost nothing else. Telling them apart is the single most useful thing to know about the term.

Few-shot prompting (in-context learning) is what is described above: examples live in the context window, no gradient is computed, no parameter changes, and the effect vanishes when the request ends. It arrived at scale with GPT-3 (Brown et al., 2020) — 175 billion parameters, 10× more than any previous non-sparse language model — whose central claim was that at sufficient scale a model can do this from examples alone. The paper packed in as many demonstrations as fit the model's 2,048-token context, typically 10 to 100, which also names the constraint: examples and the actual input compete for the same fixed budget.

Episodic few-shot training (N-way K-shot) is the classical regime, and it does update weights. Training is organised into episodes: each episode draws N classes and K labelled examples of each — the support set — plus unlabelled examples of the same classes to be scored, the query set. "5-way 1-shot" means five classes with one example each, five images in total, after which the model sorts a new image into one of those five. Because chance is 1/N, a 5-way accuracy of 65% and a 20-way accuracy of 65% are wildly different results, and a shot count quoted without its N is not interpretable. K controls how much evidence per class; N controls how hard the problem is.

The benchmark that standardised this is miniImageNet: 100 ImageNet classes at 600 images each, 60,000 images of 84×84 pixels, split by Ravi and Larochelle into 64 training, 16 validation and 20 test classes. The split is over classes, not images — the twenty test classes are never seen during training at all, which is precisely what makes the evaluation few-shot rather than ordinary supervised learning on a small dataset. The techniques built for this setting — metric learning, prototype-based classifiers, and meta-learning methods that train an outer loop over episodes — belong to it, not to prompting.

Zero-shot and one-shot sit on the same axis and have their own pages: zero-shot learning gives no examples at all, one-shot learning gives exactly one.

Real-World Applications

• Pinning output format and label space in production pipelines. This is the dominant use, and Min et al.'s result explains why it works: three examples reliably teach a classifier that the only acceptable answers are refund, exchange and escalate, which is exactly the job most extraction and routing prompts need done. It is also why a badly-behaved few-shot prompt is usually fixed by changing the shape of the examples, not by hunting for wrong labels in them.
• Low-resource machine translation. Agarwal et al. (NeurIPS 2024) pushed in-context learning into the many-shot regime — hundreds to thousands of examples in Gemini 1.5 Pro's 1M-token context — and beat the prior state of the art on translation into Bemba and Kurdish, languages with too little parallel text to fine-tune on comfortably.
• Visual classifiers for classes that barely have data. The episodic regime is the one to reach for when new categories appear faster than labels do and there is no language interface to prompt: a defect type with nine photographs, a species with a dozen. This is where the miniImageNet and Omniglot machinery earns its keep, and where "few-shot" still means training.

Key Concepts

• Demonstration: one input-output pair placed in the prompt. Not training data — it is billed as input tokens and discarded after the request.
• Support set: in the episodic regime, the N×K labelled examples the model is allowed to see for this episode.
• Query set: the unlabelled examples from the same N classes that the episode is actually scored on.
• Label space: the set of outputs the demonstrations implicitly declare to be legal. Often the most valuable thing a few-shot prompt communicates.

Challenges

The largest practical problem is that few-shot prompts are unstable in ways that look like nothing. Lu et al. (2022) showed that merely reordering the same examples can move a prompt "between near state-of-the-art and random guess performance"; their reordering method won a 13% relative improvement for GPT-family models across eleven text classification tasks without changing a single example. The search space is unhelpfully large: four examples have 4! = 24 orderings, eight have 40,320. If you evaluate one ordering and ship it, you have measured a sample of one from a distribution you have not looked at.

The random-label finding cuts the same way. If demonstrations mostly convey format and label space, then the intuitive debugging move — auditing the examples for correctness — is aimed at the wrong thing. Whether your demonstrations are representative usually matters more than whether they are right: a set of five examples where four happen to be negative will tilt the model towards predicting negative, and that skew is easy to introduce by grabbing the first few rows of a sorted file.

Then there is the cost, which is structural rather than a bug. Every demonstration is re-sent on every request, so few-shot is the only technique whose price rises linearly with success. Those tokens also have to be processed before the first output token appears, so a 2,000-token example block is a latency floor as well as a line item. And they compete directly with the input: on a long document, examples and content are drawing from the same context budget, and past a point the examples are crowding out the thing you actually wanted read.

Future Trends

• Many-shot in-context learning. With million-token contexts, "few" is becoming a historical accident. Agarwal et al. (2024) report consistent gains going from tens to hundreds and thousands of examples, and argue this makes task-specific fine-tuning less necessary — which shifts the break-even calculation above, since the token bill grows while the training bill disappears.
• Retrieved rather than fixed demonstrations. Instead of one hard-coded example block, select the examples per request by embedding similarity to the incoming input. This directly attacks the representativeness problem, at the cost of the stable prompt prefix that prompt caching depends on — the two optimisations pull against each other, and which wins depends on how repetitive your traffic is.
• Caching as the deciding variable. As cached input keeps getting cheaper relative to fresh input, the recurring cost of a fixed demonstration block falls towards a rounding error, and the argument for fine-tuning narrows to cases where prompt length itself — latency, or context crowded out — is the binding constraint rather than money.