Reasoning Model

Definition

A reasoning model is a language model trained to produce a long internal chain of working before it answers — and the practical difference from an ordinary chat model is that you pay for that working, and wait for it, on every single request. Whether the trade is worth making has a sharper answer than most vendor pages admit: reasoning models earn their premium where a cheap checker can separate a right answer from a wrong one — competition mathematics, code that passes or fails a test suite, formal constraints — and lose money on open-ended writing, summarisation, classification and retrieval, where the accuracy gain is small and the bill is not.

Here is that difference as arithmetic, using the scores OpenAI published for o1 on 12 September 2024, the launch that made the category a product. Against the non-reasoning model of the day, o1's pass@1 went from 9.3% to 74.4% on the AIME 2024 competition maths paper — an eightfold multiplication of accuracy. On MMLU, a broad multiple-choice knowledge test, the same pair went from 88.0% to 90.8%: a multiplier of 1.03. Same two models, same day, same table. The category is not "better" in some general sense; it is steeply better on a narrow shape of problem and almost flat everywhere else, and the price is charged on both.

Nothing about the architecture is new. A reasoning model is the same transformer with the same weights format behind the same API. What changed is post-training: the model was rewarded for getting checkable answers right, discovered that longer working scores better, and now writes that working every time whether you asked for it or not. Everything a reasoning model does at answer time, an ordinary model can be prompted into doing badly — which is why chain-of-thought prompting is the ancestor of this and not a substitute for it.

You can recognise one from the outside by three signs, and they matter more than the marketing name. There is a control — an effort level or a token budget — that lets you buy more or less thinking per request. There is a usage field counting reasoning or thinking tokens separately from the answer. And there is a pause: seconds to minutes before the first visible character, because the model is talking to itself first. Vendors label the same three things "reasoning effort", "extended thinking" and "thinking mode", and increasingly ship them as a mode of one general model rather than as a separate model with its own name — so "is this a reasoning model?" is turning into "is thinking on for this request?"

How It Works

The reasoning is trained in, not asked for

The mechanism that made this a product category is reinforcement learning against automatically checkable rewards. Take a base model, give it maths problems and programming problems whose answers a script can grade, and reward only the final answer. Nobody writes the reasoning; nobody demonstrates a worked solution. The model discovers on its own that backtracking, re-checking its arithmetic and abandoning dead ends raise its score, and its responses get longer over training as a result.

DeepSeek's R1 report is the clearest public account of this, because the authors ran the experiment in its purest form. Their R1-Zero variant started from a base model and applied reinforcement learning with rule-based correctness rewards and no worked examples at all. Its average pass@1 on AIME 2024 rose from 15.6% to 77.9% over training, and to 86.7% with self-consistency decoding across samples. The paper's own framing of the result is that reasoning ability "can be incentivized through pure reinforcement learning (RL), obviating the need for human-labeled reasoning trajectories." The algorithm they used to do it, GRPO, is now the default recipe for this kind of post-training.

That "obviating" is the whole economic story of the category. Supervised approaches need a human to author every reasoning chain, which does not scale. A verifier scales to as many problems as you can generate — which is exactly why the capability arrived in maths and code first, and why it has not arrived in taste.

What you actually get to control

The user-facing surface is small and it is the same everywhere: a dial. OpenAI's reasoning guide documents a reasoning_effort parameter whose supported values are model-dependent and "can include none, minimal, low, medium, high, xhigh, and max", noting that lower effort favours speed and fewer tokens while higher effort makes the model "think more completely". Anthropic's effort documentation describes the same ladder of named levels, defaults to high, and adds a detail worth internalising: effort there "affects all tokens in the response", including tool calls and function arguments — so a lower level does not merely shorten the monologue, it makes the model take fewer actions.

Older interfaces exposed a raw token budget instead — "think for at most N tokens" — and that shape is being retired in favour of named levels that the model interprets adaptively. Either way, the thing you are setting is not a quality slider. It is a purchase order for test-time compute, placed per request, by whoever writes the API call.

What you cannot see, and pay for anyway

The reasoning itself is withheld. OpenAI's guide states it twice, in two places, without hedging: "While reasoning tokens are not visible via the API, they still occupy space in the model's context window and are billed as output tokens", and "we don't expose the raw reasoning tokens emitted by the model" — you may request a summary instead. The volume is not small: the same guide says models "may generate anywhere from a few hundred to tens of thousands of reasoning tokens" depending on the problem.

Three consequences follow, and each of them has broken somebody's production system:

• Your bill is not your transcript. The tokens you are charged for are strictly more than the tokens you can read, and no billing line separates them.
• Reasoning competes with your answer for space. It counts against the output cap and against the context window. Set the cap too low and the model spends the entire allowance thinking and gets truncated mid-sentence.
• Debugging is inference. When a reasoning model returns something wrong, the thing that produced it is not in the response. You are reasoning about a black box from its summary — and the summary is a reconstruction, not a log. See chain-of-thought for how far apart those two can be.

The one calculation that decides whether to use one

The decision is not "is this model smarter". It is "does the extra accuracy come cheaper than the extra price", and that has a one-line answer.

Let k be the cost multiple — what one reasoning answer costs relative to one direct answer, including the hidden tokens. Let a_r and a_b be the accuracies you measure on your task. Cost per correct answer is price divided by accuracy, so the reasoning model is the cheaper way to buy a correct answer exactly when:

k < a_r / a_b

The right-hand side is the break-even multiple, and it is fixed by the task, not by the vendor. Run it on the September 2024 o1 table and the gradient is stark:

Read the last row carefully, because it is the row most production workloads actually resemble. To be worth it on MMLU-shaped work, a reasoning model would have to cost less than 3% more per question. Nothing that emits thousands of hidden tokens costs 3% more. On AIME-shaped work it could cost eight times as much and still be the cheaper route to a right answer.

Two honest caveats. The multiplier k is the one number you cannot look up — it depends on how many hidden tokens your prompts provoke, and you get it by running your own traffic and reading the usage fields. And accuracy is not the only axis: a task where a wrong answer is expensive to clean up justifies a worse cost-per-correct-answer than one where you can just retry.

Real-World Applications

Agentic coding is where the category is genuinely deployed at scale, and the vendors' own benchmark footnotes show why — and how modest the per-task effect can be. Anthropic's launch footnote for Claude Haiku 4.5 (October 2025) reports Terminal-Bench scores "averaging 11 runs (6 without thinking (40.21% score), 5 with 32K thinking budget (41.75% score))". Same model, same harness, one variable: +1.54 points for a 32,000-token thinking budget. That is a real gain on a hard agentic benchmark and it is also a reminder that thinking is a multiplier on an existing capability, not a substitute for one. Note also what the footnote implies about every other score you read: the headline number was produced with a thinking budget enabled, so an out-of-the-box call with thinking off will not reproduce it. (This is the same problem state-of-the-art claims have generally.)

The effort dial has become a routing decision inside products. A support assistant answering "where is my order" and the same assistant asked to reconcile a disputed invoice do not need the same amount of thinking, and the API now lets one system make that call per request. That is a design decision someone makes weekly: classify the incoming request, pick an effort level, accept that the cheap path is occasionally wrong and the expensive path is occasionally slow enough that the user leaves.

Reasoning traces are training data for smaller models. DeepSeek's R1 report showed that supervised fine-tuning of a mid-size dense model on 800,000 samples generated by R1 beat running large-scale RL on that same base model directly — the chains of a big reasoner become the supervision signal for a small one. This is why capable small models exist at all, and it belongs to knowledge distillation.

Where they are deliberately not used: high-volume classification, retrieval-augmented question answering over your own documents, summarisation, and anything user-facing that must feel instant. In RAG, the accuracy of the answer is mostly decided by whether the right passage was retrieved; a model that deliberates for thirty seconds over the wrong passage produces a confident wrong answer more slowly and more expensively than one that does it immediately.

Key Concepts

• Verifier, not difficulty, predicts the gain. The question is never "is this task hard" but "can something cheap tell a good attempt from a bad one". Hard-and-uncheckable — a strategy memo, a piece of prose — is the quadrant where the category has the least evidence.
• Effort is a purchase, not a quality setting. Every level you raise buys tokens at output rates and seconds of latency. Treat it as a line item you tune per route, not a default you set once and forget.
• pass@1 is the product; cons@64 is the poster. Several of the loudest reasoning-model numbers were obtained by sampling dozens of times and letting a voter or an oracle pick — a configuration you will not run in production.
• The trace is not a transcript. What you are shown is a summary of what you are billed for, and even the raw chain is a plausible account of the answer rather than a record of the machinery that produced it.

Challenges

They do not help on the things most people write prompts for, and the vendors have said so. OpenAI's own o1 launch reported a human-preference evaluation on open-ended prompts and published the awkward half of the result: o1-preview "is preferred to gpt-4o by a large margin in reasoning-heavy categories like data analysis, coding, and math. However, o1-preview is not preferred on some natural language tasks, suggesting that it is not well-suited for all use cases." That is the launch post, not a critic. If your workload is drafting, rewriting, tone, or summarisation, the reasoning premium buys latency and a bigger invoice.

Latency is a product change, not a configuration change. A model that streams its first token in 300 milliseconds and one that thinks for forty seconds are different products even when both are correct, and no amount of spinner design makes them the same. This is the failure that survives contact with users: teams evaluate on accuracy, ship, and discover that the thing they broke was the interaction. If you cannot restructure the interface around waiting — background jobs, progress updates, asynchronous delivery — a reasoning model may be the wrong choice on a task where it is measurably more accurate.

What breaks when you get this wrong is specific and it is always the same three things. A team benchmarks a reasoning model on a sample, sees it win, and rolls it out on a high-volume workload with no verifier — ticket triage, tagging, summarising. The bill arrives several times over the estimate, because the estimate counted the visible answer and the invoice counted the hidden thinking. Latency multiplies. And accuracy does not move, because there was never a checker to make the extra tokens pay. All three come from the same mistake: treating the effort dial as a quality setting rather than as a purchase with a break-even point.

Output caps are the sharpest operational edge. Reasoning tokens count against the same ceiling as your answer. A max_tokens value tuned for a chat model will, on a hard question at high effort, be consumed almost entirely by thinking — and the response you get back is a truncation, not an error you can catch by type. Leave headroom proportional to the effort level, and check the stop reason.

Longer is not monotonically better. More deliberation has diminishing returns everywhere and negative returns in places; over-instructing a reasoning model to reason — pasting a 2022 "let's think step by step" template onto a 2026 model — is one of the reliable ways to land on the wrong side of that curve. Measure it on your own task instead of assuming direction.

And the name oversells. "Reasoning" describes a training objective and an output shape, not a faculty. The model was rewarded for producing token sequences that end in checkable-correct answers. On problems that look like the ones it was rewarded on, that is enormously effective. Read it as evidence of general judgement and you will be surprised in exactly the domains where no verifier existed to keep it honest.

Future Trends

The clearest trend is that the category is dissolving into a parameter. The first reasoning models shipped as separate models with separate names and separate prices. The current shape is one model with a thinking mode and an effort ladder, sometimes on by default, sometimes decided adaptively by the model itself. If that continues, "reasoning model" stops naming a product and starts naming a request configuration — which is a better description of what it always was.

The second is routing: a cheap model or a classifier decides, per request, how much thinking to buy, so that the eightfold-payoff queries get deliberation and the flat ones do not. That turns the break-even calculation above from a design-time decision into a runtime one, and it makes the quality of the router — a component nobody benchmarks — a first-order determinant of both cost and accuracy.

The binding constraint on both is verification. Everything this category is reliably good at sits downstream of a cheap correctness check, because that check is what made the training possible and what makes the extra tokens pay. Extending it to domains where correctness is contested means building verifiers that are themselves models, with their own errors and their own exploitable seams. Whether that works decides whether reasoning models keep generalising or settle permanently at the boundary of what a unit test can confirm.

Code Example

The break-even rule is three lines of arithmetic, and running it on your own measured numbers is the single most useful thing you can do before adopting a reasoning model. The rows below are OpenAI's published pass@1 figures from September 2024 — historical, used here for the shape of the result, not as a claim about any current model. Replace them with your own evaluation set.

# pass@1 scores published by OpenAI on 12 September 2024 for gpt-4o and o1.
# Historical snapshot: use the shape, not the values.
BENCHMARKS = [
    ("AIME 2024 (competition maths)", 9.3, 74.4),
    ("MATH", 60.3, 94.8),
    ("GPQA Diamond", 50.6, 77.3),
    ("MMLU", 88.0, 90.8),
]

COST_MULTIPLE = 7.0   # what one reasoning answer costs vs one direct answer

print(f"{'benchmark':<32} {'break-even':>10} {'verdict at 7x':>14}")
for name, direct, reasoning in BENCHMARKS:
    break_even = reasoning / direct
    verdict = "worth it" if COST_MULTIPLE < break_even else "loses money"
    print(f"{name:<32} {break_even:>9.2f}x {verdict:>14}")

Output:

benchmark                        break-even  verdict at 7x
AIME 2024 (competition maths)         8.00x       worth it
MATH                                  1.57x    loses money
GPQA Diamond                          1.53x    loses money
MMLU                                  1.03x    loses money

One row survives. That is not an argument against reasoning models — it is the argument for knowing which row your task is on before you pay. COST_MULTIPLE is the number to replace first: it is not a published figure, it is the ratio of total billed tokens (thinking plus answer) to answer tokens on your prompts, which you read out of the usage fields on a few hundred real requests. Where that ratio comes from and why it is so often larger than teams expect belongs to test-time compute.