Definition
Prompt engineering is the practice of writing the input you send to a large language model so that the model reliably produces the output you want. The key thing to hold onto is that the model is already trained and fixed: prompt engineering changes nothing inside it — no weights, no memory — it changes only the words going in. You are steering a car, not rebuilding the engine.
That is what separates it from fine-tuning, the other way to change a model's behaviour. Fine-tuning runs a training job that edits the model's weights and needs a labelled dataset; prompt engineering edits a text box and takes effect on the very next request. One is surgery, the other is conversation. Because prompting is instant and free to try, it is almost always the first lever you reach for — and for most tasks it is the only one you need.
The reason a few words in front can move so much is worth stating plainly, because it is also the most measurable fact in the field. In the 2022 paper that named chain-of-thought prompting, Google's PaLM 540B model solved 17.9% of the GSM8K grade-school maths problems when asked for the answer directly, and 56.9% when the same prompt asked it to show its working first — a jump of about 39 points. Identical model, identical questions; the only thing that changed was the instruction. Prompt engineering is the discipline of finding changes like that on purpose instead of by luck.
How It Works
Underneath, a language model is a fixed function: it reads your text as tokens and predicts the next token, then the next, one at a time. At inference the weights are frozen, so the only thing you control is the sequence of tokens the model reads before it starts answering. The prompt is that sequence. Everything below is a way of arranging those tokens so the model's next-token predictions land where you want them.
Clear, specific instructions do most of the work. The model cannot ask a clarifying question, so any ambiguity gets resolved by whatever pattern was most common in training — usually not what you meant. "Summarise this" invites a summary of unknown length, audience and format; "Summarise this in three bullet points for a non-technical manager, each under 15 words" removes the guesses. Specificity is not politeness, it is constraint: each detail you pin down is a branch of the model's output distribution you have cut off.
Giving examples teaches a pattern the instruction cannot describe. When a task is easier to show than to state — a particular JSON shape, a tone of voice, a labelling scheme — you paste a few worked input-output pairs into the prompt itself. This is few-shot prompting, and its defining feature is that it updates no weights: the examples are read, used to condition the answer, and discarded when the request ends. It sits on a spectrum from zero-shot (instruction only) through one-shot to few-shot; the few-shot page works through when the extra examples pay for themselves and when they stop helping, so this page does not repeat that arithmetic.
Chain-of-thought asks the model to reason before it answers. Adding a phrase like "Let's think step by step", or showing examples that include the reasoning and not just the answer, makes the model generate intermediate steps first. The mechanism is mechanical rather than mystical: each step the model writes becomes part of the context for the next step, so a hard problem is broken into a chain of easy next-token predictions instead of one impossible leap to the final number. That is where the 17.9%-to-56.9% GSM8K gain above comes from. The catch is that it is an ability of scale — below roughly 100B-parameter models the same prompt produces fluent but wrong reasoning — so the chain-of-thought page is worth reading before relying on it.
Structure and delimiters stop the model from mixing up its inputs. When a prompt contains both
instructions and the data to act on, wrapping the data in explicit markers — triple backticks, XML
tags, a labelled heading — tells the model where one ends and the other begins, so it does not treat
your document as a new instruction. The same move works on the output: asking for a named format
("return a JSON object with keys title and summary") turns a free-text reply into something a
program can parse without a fragile regular expression. A worked before-and-after looks like this:
Weak prompt (instruction and data blur together):
Translate to French and make it formal: hey can you send the report
Structured prompt (delimited input, specified output):
Translate the text between <text> tags into formal French.
Return only the translation, no commentary.
<text>hey can you send the report</text>
System prompts set the frame; user prompts carry the request. Most chat models accept two kinds of message. The system prompt is read first and establishes standing rules — role, tone, what the model may and may not do — while user prompts are the individual turns. Because the system prompt conditions everything after it, it is where durable behaviour belongs ("You are a support agent for Acme; never promise refunds"), leaving the user turn to carry only the specific task. Putting a one-off instruction in the system prompt, or a standing rule in every user turn, is a common source of inconsistent behaviour.
None of these is a rule to apply blindly. Prompt engineering is empirical: you write a candidate, run it on real inputs, look at where it fails, and adjust. The techniques above are where to look first, not a checklist to complete.
Real-World Applications
Prompt engineering is the layer that turns a general model into a specific product, and the clearest examples are the ones you can inspect. Coding assistants like GitHub Copilot Chat and Cursor wrap every request in a hidden system prompt that supplies the open file, the surrounding code and instructions about how to format a diff — the same underlying model behaves like a pair programmer only because of what is prepended to your question. Customer-support and agent products ship long, carefully versioned system prompts that define the assistant's role, its refusal boundaries and its escalation rules; providers such as Anthropic and OpenAI publish prompt-engineering guides precisely because these prompts are the main thing a team building on the API actually writes and maintains.
The highest-leverage everyday use is structured extraction: asking a model to read messy text — an invoice, a support email, a CV — and return a fixed JSON schema. Here the format instruction and delimiters are not stylistic, they are what makes the output usable by the next program in the pipeline, and OpenAI's structured-outputs and JSON-mode features exist to make that instruction reliable. In retrieval-augmented generation, the prompt is where fetched documents get inserted with instructions to answer only from them, which is a prompt-engineering decision that directly controls how often the system makes things up.
One boundary is worth drawing. Prompt engineering aims your input at the model; generative engine optimization aims your published content at models that other people query, so that an AI answer engine cites you. They share vocabulary and nothing else — one is about the prompt you write, the other about being the answer to someone else's prompt.
Challenges
The failures in prompt engineering are quiet ones — nothing throws an error, the output just gets worse — which is what makes them worth naming.
A prompt is not portable, and the degradation is silent. A prompt tuned against one model encodes assumptions about how that model reads instructions and formatting. Move it to a different model, or even a new version of the same one, and it may quietly lose accuracy with no warning: the call still succeeds and returns fluent text, it is just wrong more often. This is why a prompt is not a write-once asset — every model upgrade is a reason to re-test the prompts that depended on the old one.
Examples are a recurring tax, not a one-time cost. Every few-shot example lives in the context window and is re-sent and re-billed on every call, forever. A generous ten-example block that improves quality by a couple of points can, on a high-traffic endpoint, cost more in cumulative tokens than the improvement is worth — and because the examples work, nobody goes back to trim them. The few-shot page treats this trade-off directly; the discipline is to use the fewest examples that hold the quality, not the most that fit.
Untrusted text can hijack the instructions. Because the model treats its whole context as one stream of tokens, text you paste in — a web page, an email, a document — can contain instructions of its own ("ignore your previous instructions and…") that the model may follow. This is prompt injection, and it has no clean fix: delimiters and system-prompt reminders reduce it but do not close it, so any system that feeds user-supplied content into a prompt has to assume the content is adversarial.
The output is non-deterministic, so evaluation is hard. The same prompt can give different answers on different runs, which means you cannot judge a prompt from one lucky result. Serious prompt work needs a small evaluation set — a handful of representative inputs with known-good outputs — run repeatedly, or you are tuning against noise and shipping a prompt that happened to work the one time you looked.
Future Trends
The clearest trend is that hand-tuning is being partly automated. Frameworks now treat a prompt as something to optimise rather than handwrite, searching over instructions and example sets against an evaluation metric — the same idea as tuning any other parameter, applied to text. In parallel, reasoning models increasingly bake in behaviour that used to require prompting: where you once added "think step by step", newer models produce the reasoning on their own, shifting the skill from eliciting a behaviour to knowing when it is worth the extra tokens. A third trend renames the discipline: once the window fills with tool schemas, retrieved documents and a growing transcript, the job becomes budgeting that space rather than wording a request, which is what context engineering describes. None of these removes the core job, which is deciding precisely what you want and expressing it unambiguously — that part is upstream of any model and does not automate away.