Term tag: Tokens

Chunking

Chunking splits documents into smaller passages before embedding them for retrieval. Chunk size, overlap, and splitting strategy quietly decide RAG quality.

Context Engineering

Deciding what enters a model's context window each turn and what gets evicted — the discipline that took over from prompt engineering once agents arrived.

Context Window

Everything a model can see at once — why the advertised million tokens is not the usable number, and why you pay for the whole window on every turn.

Large Language Model (LLM)

A large language model predicts the next token of text. How that produces answers, where its knowledge is stored, and why an LLM is not a chatbot.

Multimodal AI

How one model reads an image and text at once: the picture is cut into fixed patches, each patch becomes a token, and those tokens cost context.

Structured Outputs (Constrained Decoding)

Constrained decoding masks a language model's token distribution against a JSON schema or grammar at each step, so the output is parseable by construction.

Token

The unit an AI model reads, writes and bills by — roughly 1.3 tokens per English word. Input, output and cached tokens are priced up to 10x apart.

Tokenization

How text becomes the integers a language model reads: BPE, WordPiece and Unigram, why " strawberry" is one token, and what that breaks.