Term tag: Long Context

Agent Memory

Agent memory is how an AI agent stores, retrieves, and summarizes information so it works across long tasks and sessions without overflowing its context window.

Context Engineering

Deciding what enters a model's context window each turn and what gets evicted — the discipline that took over from prompt engineering once agents arrived.

Context Window

Everything a model can see at once — why the advertised million tokens is not the usable number, and why you pay for the whole window on every turn.

FlashAttention

The exact-attention algorithm that made long context affordable: it never writes the n x n score matrix to GPU memory, so it runs faster and changes no output.

Grouped-Query Attention (GQA)

Grouped-query attention lets query heads share key/value heads, shrinking the KV cache that dominates long-context inference at near multi-head quality.

KV Cache

The memory an LLM keeps so it never recomputes the past — why a long context costs money, and how many users one GPU can actually serve at once.

Positional Encoding (RoPE)

Positional encoding tells a transformer where each token sits in a sequence. Why self-attention needs it, and how rotary embeddings (RoPE) work.

State-Space Models (Mamba)

State-space models are sequence models that scale linearly with length, not quadratically like attention. Mamba's input-dependent state made them competitive.