---
source: 'https://howaiworks.ai/glossary/token'
section: glossary
title: Token
description: >-
  The unit an AI model reads, writes and bills by — roughly 1.3 tokens per
  English word. Input, output and cached tokens are priced up to 10x apart.
tags:
  - Tokens
  - LLM
  - NLP
  - Inference
  - AI Infrastructure
category: Natural Language Processing
datePublished: '2025-09-01'
lastUpdated: '2026-09-06'
---

# Token

> The unit an AI model reads, writes and bills by — roughly 1.3 tokens per English word. Input, output and cached tokens are priced up to 10x apart.

## Definition

A token is one entry from a model's fixed vocabulary — most often a fragment of a word — and it is
the only thing an AI model ever sees. Your text is converted into a list of integers before the
model runs, the model does arithmetic on those integers, and integers come back out. The words you
read were never in the building.

That conversion is not a detail, because **everything is counted in tokens**: what you pay, what
fits in the [context window](https://howaiworks.ai/glossary/context-window), how much the model may write back, and how
much GPU memory your session occupies while you hold it. Measured across the 437,000 words of
prose in this site's glossary with OpenAI's `o200k_base` encoder, the exchange rate is **4.65
characters per token, or 1.29 tokens per English word** — so 1,000 words costs roughly 1,290
tokens. The old rule of thumb that a token is four characters, which comes from GPT-3's tokenizer,
survives better than it deserves to: it understates a modern encoder by about 15%, and the
companion rule that a token is three-quarters of a word lands within 3% of the measurement.
Rerun it on your own text before trusting either, because the average is corpus-shaped — the same
encoder returns 1.35 tokens per word on this site's blog and 1.07 on plain unformatted English.

This page is about the units and what they cost. [Tokenization](https://howaiworks.ai/glossary/tokenization) is about
how the vocabulary that produces them gets built.

## How It Works

A token id is an index. The model holds an embedding table with one row per vocabulary entry, so
token 101830 means "fetch row 101830" — a vector of a few thousand numbers that is the model's
learned representation of that fragment. Every layer transforms that vector in the context of its
neighbours, and the final layer produces a score for *every* entry in the vocabulary. Sampling one
of them yields the next token, which is appended to the sequence, and the whole thing runs again.

**That loop is the reason output costs more than input.** A prompt of any length is processed in a
single pass, because all its tokens are known at once and the arithmetic parallelises across them.
Output has no such luxury: token 400 cannot be computed until token 399 exists, so a 500-token reply
means 500 sequential passes through every layer of the model. The pricing reflects the physics —
[GPT-6 Astra](https://howaiworks.ai/models/gpt) charges **$10 per million input tokens against $50 per million output**,
[Claude Opus 5](https://howaiworks.ai/models/claude-opus) $5 against $25, and [Gemini 3.8 Flash](https://howaiworks.ai/models/gemini) $0.75
against $3.75 on its introductory rate. Three vendors, a 13x spread in absolute price, and the
identical **5:1 ratio** in all three, because they are all paying for the same asymmetry. The
outliers are informative rather than contradictory: [Grok 4.6](https://howaiworks.ai/models/grok) at $2 against $6 for
prompts under 200,000 tokens is 3:1, which tells you it is priced to win high-output agent work.

**Tokens also occupy memory for as long as you keep them.** Each token already processed leaves
behind cached key and value vectors so it does not have to be recomputed — the
[KV cache](https://howaiworks.ai/glossary/kv-cache). For Llama 3.1 70B that costs 320 KiB per token, so a single user at
a 128K context is holding **42.9 GB of GPU memory**, more than half an H100. A token is not a
character with a price tag; it is a character with a price tag and a rent.

## Types

The categories below are not a conceptual scheme — they are the line items on an API invoice, and
mistaking one for another is how estimates go wrong by an order of magnitude.

### Input tokens

Everything you send: system prompt, conversation history, retrieved documents, tool definitions and
tool results. In a multi-turn conversation the entire history is resent on every request, so input
grows quadratically with turn count even though each individual message is small.

### Output tokens

Everything the model generates, billed at 3-6x the input rate and most often at exactly 5x.
Providers cap them separately from the context window, and often cap input separately too:
GPT-6 Astra has a 1,050,000-token context of which at most **922,000 may be input**, and it will
emit at most **128,000** whatever room is left. The three numbers are three different limits and only one of them is the headline.

### Reasoning tokens

Tokens a reasoning model produces for itself before answering. They are billed as output, they
consume the output cap — on GPT-6 Astra the same 128,000 — and they are not returned to you. A
request can therefore fail to produce a visible answer while having been charged in full, which is a
failure mode that did not exist before reasoning models.

### Cached tokens

Input the provider has already processed and can replay cheaply. GPT-6 Astra bills cached input at
**$1.00 per million against $10.00** — a 10x discount — provided the prefix is byte-identical. It
also charges **$12.50 per million to write** the cache, which is 1.25x the uncached rate, so the
discount is a loan you repay on the first call and collect on every one after. Gemini 3.8 Flash
prices caching as rent rather than as a discount: $0.075 per million tokens read, plus $0.50 per
million tokens per hour of storage. Anthropic sits between the two, with cache reads at 0.1x the
base input rate and a 5-minute write at 1.25x.

### Special and control tokens

Vocabulary entries that mean something structural rather than lexical. BERT uses `[CLS]`, `[SEP]`,
`[MASK]` and `[PAD]`; chat models use control tokens such as `<|im_start|>` to mark where the system
prompt ends and the user's turn begins. This is why a chat request is not simply your strings
concatenated: the template inserts control tokens, and they are billed like any other.

### Tokens that are not text

Images, audio and video are converted to tokens too, and billed as input. The count tracks
resolution — [Claude Sonnet 5](https://howaiworks.ai/models/claude-sonnet) raised its limit from 1568 to 2576 pixels on
the long edge, and a high-resolution image can consume roughly **3x more image tokens** as a result.
A screenshot pasted into a prompt is not free context.

## Real-World Applications

**Which side of the invoice dominates depends entirely on the workload, and it flips.** Take
[GPT-6 Astra](https://howaiworks.ai/models/gpt) at $10 in and $50 out per million, and two jobs of similar size:

| Job | Input | Output | Input cost | Output cost | Bill |
|---|---|---|---|---|---|
| Summarise a 50,000-token report | 50,000 | 500 | **$0.500** (95%) | $0.025 | $0.525 |
| Coding agent writes a module | 2,000 | 4,000 | $0.020 | **$0.200** (91%) | $0.220 |

The first row is 50,000 × $10 ÷ 1,000,000 = $0.500 of input against 500 × $50 ÷ 1,000,000 = $0.025
of output: a $0.525 bill that is 95% prompt. The second is $0.020 against $0.200 — a $0.220 bill
that is 91% reply. Same model, same price list, opposite conclusion about what to optimise.
Shortening prompts is wasted effort on the second job; shortening replies is wasted effort on the
first. You cannot know which you are running without counting both sides.

**Caching changes the arithmetic more than model choice does.** Ask twenty questions about the same
50,000-token document and you send a million input tokens: 1,000,000 × $10 ÷ 1,000,000 = **$10.00**
at Astra's full rate. Cache the prefix instead and the first call pays the *write* rate,
50,000 × $12.50 ÷ 1,000,000 = $0.625, while the remaining nineteen read it at
950,000 × $1.00 ÷ 1,000,000 = $0.950. Total **$1.575 — a 6.3x saving** on an identical workload.

The write rate is the part people miss. At $12.50 per million it is 1.25x the uncached price, so
caching a prefix you use exactly once costs $0.625 where sending it plainly costs $0.500: you have
paid 25% extra for a discount you never collect. Caching pays from the second call onward, and only
while the prefix stays byte-identical — which is why injecting a timestamp at the top of a system
prompt can quietly multiply a bill by six.

**Token counts are what specification sheets actually specify.** "1M context" means 1M tokens, not
characters or words: at 1.29 tokens per word that is about **775,000 English words**, seven or
eight full-length novels. Read the sub-limits too. Astra's 1,050,000-token context admits at most
922,000 input tokens — about 715,000 words — and reserves the rest for a reply capped at 128,000.

**And the ceiling is not always flat.** Astra prices any request whose input exceeds **272,000
tokens** at 2x the input and cache rates and 1.5x the output rate — **for the entire request, not
just the tokens above the line.** That distinction is the whole lesson. A 271,000-token prompt bills
at 271,000 × $10 ÷ 1,000,000 = **$2.71** of input. Add two thousand tokens and the 273,000-token
prompt reprices end to end at $20 per million: 273,000 × $20 ÷ 1,000,000 = **$5.46**. Those 2,000
extra tokens — 0.7% more input — cost $2.75 between them, an effective **$1,375 per million, or
137.5x the sticker rate**. Output crosses at the same instant, from $50 to $75 per million, and cache
reads from $1.00 to $2.00.

The practical consequence is that a prompt hovering near the line is worth trimming rather than
tolerating, and a retrieval step that *sometimes* returns 280,000 tokens has a bimodal bill nobody
budgeted for. The shape is not unique to OpenAI: [Grok 4.6](https://howaiworks.ai/models/grok) reprices the whole request
at or above 200,000 tokens, and GPT-5.6 carries Astra's rule at the same 272,000-token threshold.

## Challenges

**The invisible half of the bill.** Reasoning tokens are charged at the output rate and never shown.
A request that looks like it produced 200 tokens may have been billed for 8,000, and the only way to
see it is the usage field in the API response — not the reply. Any cost model built by measuring
visible output is wrong for every reasoning model.

**Conversation history is a compounding cost.** Because the full transcript is resent each turn, a
chat that reaches 50 turns of 500 tokens has sent roughly 640,000 input tokens, not 25,000. Long
agent loops fail on budget for this reason far more often than they fail on capability.

**The rate card is not a constant, and it moves both ways.** When Astra launched on 4 September 2026
OpenAI cut the previous generation's prices underneath it: GPT-5.6 Sol went from $5/$30 per million
to **$4/$20** — 20% off input and 33% off output — and GPT-5.6 Luna from $1/$6 to **$0.20/$1.20**,
an 80% cut on both sides. A unit-economics model written in August overstated a September bill by a
fifth to a third, and the cut left GPT-5.5, untouched at $5/$30, costing *more* than the newer model
that supersedes it. Prices move up too: Gemini 3.8 Flash's $0.75/$3.75 is an introductory rate that
doubles to $1.50/$7.50 on 1 January 2027. Treat any per-token figure, including the ones on this
page, as a reading rather than a constant.

**Counting with the wrong tokenizer.** Each model family has its own vocabulary, and a count from
one does not transfer to another. Estimating a Claude bill with `tiktoken` produces a number that is
confidently wrong, and the error is largest exactly where it matters: non-English text and code.

**Context limits are not word limits.** A 128,000-token output cap sounds generous until a task
emits JSON, where structural characters, quoted keys and escaped strings drive the token count far
above what the equivalent prose would cost. The same content in a compact format can fit where the
verbose one truncates.

**Truncation is silent.** Exceeding an output cap does not raise an error — it returns a response
that simply stops, often mid-structure. Code that parses model output without checking the finish
reason will read a truncated reply as a complete one.

## Code Example

Counting is cheap, estimating is not. Count — and price all four rates, not two.

```python
import tiktoken

enc = tiktoken.get_encoding("o200k_base")     # GPT-4o / GPT-5 / GPT-6 family

prompt = open("system_prompt.txt").read()
n_in = len(enc.encode(prompt))

# GPT-6 Astra, dollars per million tokens.
IN, OUT, CACHE_READ, CACHE_WRITE = 10.00, 50.00, 1.00, 12.50
CLIFF = 272_000        # above this the WHOLE request reprices, not just the overage
expected_out = 800

over = n_in > CLIFF
m_in = 2.0 if over else 1.0        # input and cache rates double
m_out = 1.5 if over else 1.0       # output goes to 1.5x

print(f"input   {n_in:>7} tokens  ${n_in / 1e6 * IN * m_in:.4f}")
print(f"output  {expected_out:>7} tokens  ${expected_out / 1e6 * OUT * m_out:.4f}")

# Twenty calls sharing this prefix: uncached, versus one cache write and nineteen reads.
uncached = 20 * n_in / 1e6 * IN * m_in
cached = n_in / 1e6 * CACHE_WRITE * m_in + 19 * n_in / 1e6 * CACHE_READ * m_in
print(f"20 calls: ${uncached:.4f} uncached vs ${cached:.4f} cached")
```

On a 50,000-token prompt that prints `$0.5000` of input, `$0.0400` of output, and
`20 calls: $10.0000 uncached vs $1.5750 cached` — the same arithmetic worked above. Push `n_in`
past 272,000 and every one of those figures changes, which is the point of keeping the threshold in
the code rather than in your head.

For any model not made by OpenAI, load its own tokenizer instead —
`AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B")` from `transformers` — and read the
`usage` block of the API response afterwards to see what you were actually billed, including the
reasoning tokens no count can predict in advance.

---

*Note: the prices on this page were last verified on 6 September 2026 against OpenAI's GPT-6 Astra
model reference, Anthropic's API pricing page and Google's Gemini pricing. Per-token rates, context
limits and surcharge thresholds are the parts of this page most likely to age; the mechanisms they
illustrate are not.*

## Frequently Asked Questions

### What is a token in AI?

A token is one entry from a model's fixed vocabulary — usually a word fragment — and it is the only thing the model ever sees. Text is converted to a list of token ids before the model runs, and everything is counted in those ids: the price, the context limit and the output cap.

### How many tokens is a word?

About 1.3 tokens per English word, or roughly 1,290 tokens per 1,000 words. Measured across the 437,000 words of prose in this site's glossary with OpenAI's o200k_base encoder, the average is 4.65 characters per token. Code, names and non-English text run higher.

### Why do output tokens cost more than input tokens?

Because they are produced differently. An entire prompt is processed in one parallel pass, while each output token requires its own full forward pass through the model. GPT-6 Astra charges $10 per million input tokens and $50 per million output tokens — a 5:1 ratio, and Claude Opus 5 and Gemini 3.8 Flash both land on the same ratio at different absolute prices.

### What are reasoning tokens?

Tokens a reasoning model generates internally before it answers. They are billed at the output rate and consume the output budget — on GPT-6 Astra they count against the same 128,000 token cap as the visible reply — but they are not returned in the response.

### Do images count as tokens?

Yes. Images are converted to tokens and billed as input, with the count driven by resolution. Claude Sonnet 5 raised its limit from 1568 to 2576 pixels on the long edge, and a high-resolution image can consume roughly 3x more image tokens as a result.

### Why does adding a timestamp to a system prompt make it more expensive?

Because caching only fires on a byte-identical prefix. Send a 50,000-token document twenty times to GPT-6 Astra and you pay $10.00 at the $10-per-million input rate; cached, you pay one $0.625 cache write plus nineteen reads at $1.00 per million for $1.575 in total, a 6.3x saving. A timestamp at the top changes the first bytes on every call, so nothing matches and the discount silently disappears.

## Related

### Related terms

- [Tokenization](https://howaiworks.ai/glossary/tokenization)
- [Context Window](https://howaiworks.ai/glossary/context-window)
- [KV Cache](https://howaiworks.ai/glossary/kv-cache)
- [Embedding](https://howaiworks.ai/glossary/embedding)
- [Large Language Model (LLM)](https://howaiworks.ai/glossary/large-language-model)
- [Inference](https://howaiworks.ai/glossary/inference)

---

Source: https://howaiworks.ai/glossary/token — HowAIWorks.ai