Term tag: LLM

Agent Memory

Agent memory is how an AI agent stores, retrieves, and summarizes information so it works across long tasks and sessions without overflowing its context window.

AI Agent

An AI agent is an LLM-powered system that plans, calls tools, and acts autonomously in a loop to finish multi-step tasks — not just answer questions.

AI Architecture

How a production AI system is assembled: retrieval, context assembly, the model call, tool execution, validation, logging and evaluation.

AI Energy Consumption

How much energy an AI query really uses, how training compares with serving, and how to read the numbers without believing either the hype or the panic.

AI Hallucinations

Models hallucinate because training and benchmarks reward guessing over admitting ignorance: a guess sometimes scores, "I don't know" scores zero every time.

Chain-of-Thought (CoT)

A model writing out its reasoning before answering. Once a prompting trick, now trained in by RL, billed as output tokens, and frequently unfaithful.

Computer Use (GUI Agents)

How AI operates software by reading screenshots and issuing clicks and keystrokes: the perception-action loop, why grounding is hard, and what OSWorld measures.

Context Engineering

Deciding what enters a model's context window each turn and what gets evicted — the discipline that took over from prompt engineering once agents arrived.

Context Window

Everything a model can see at once — why the advertised million tokens is not the usable number, and why you pay for the whole window on every turn.

Conversational AI

Conversational AI holds a multi-turn dialogue in natural language. Unlike an intent-based chatbot, it generates each reply rather than picking one.

Diffusion Language Models (DLMs)

Text models that unmask many tokens per pass instead of writing one at a time — what that actually buys in speed, and what it costs in quality.

Direct Preference Optimization (DPO)

DPO trains a model on human preferences without a reward model or reinforcement learning: the derivation, the memory it saves, and where it loses to PPO.

Disaggregated Serving (Prefill/Decode)

Disaggregated serving runs LLM prefill and decode on separate GPU pools because the two phases hit opposite limits: compute versus memory bandwidth.

Educational AI

AI that tutors, adapts practice and grades work — and what forty years of effect-size research actually shows about whether it improves learning.

Error Handling in AI Systems

Ordinary code fails by raising an exception. A model fails by returning a confident, well-formed, wrong answer that nothing catches. What to do about it.

Few-shot Learning (FSL)

Teaching a model a task by showing it a handful of worked examples. In its modern form, few-shot prompting, no weights are updated at all.

Fine-tuning (FT)

What fine-tuning changes inside a model, what it costs in GPU memory, and why it reliably buys format and style but not new facts.

FlashAttention

The exact-attention algorithm that made long context affordable: it never writes the n x n score matrix to GPU memory, so it runs faster and changes no output.

Function Calling (Tool Calling)

The model never runs your function. It emits a structured request; your code decides whether to honour it, executes it, and hands the result back.

Generative AI

Generative AI learns the probability distribution its training data came from, well enough to draw new samples from it — not just sort inputs into classes.

Generative Engine Optimization (GEO)

GEO — also called AEO — is optimizing content so AI answer engines cite it. From a 2024 KDD paper that found keyword stuffing backfires.

GraphRAG

GraphRAG runs RAG over a knowledge graph and community summaries built from a corpus, answering global questions that plain vector RAG cannot.

Group Relative Policy Optimization (GRPO)

GRPO is PPO without the critic: it scores a group of sampled answers against each other. How it works, what it saves, and where it biases training.

Grouped-Query Attention (GQA)

Grouped-query attention lets query heads share key/value heads, shrinking the KV cache that dominates long-context inference at near multi-head quality.

Guardrails

A guardrail is a deterministic check around a model that can block, rewrite, or force a retry — external code you can test, unlike prompting a model to behave.

Inference

Running a trained model to get an answer. Training is paid once; inference is paid on every request — and it is where most AI money goes.

Inference Optimization

How a served LLM is made cheap: continuous batching, the prefill/decode split, and the throughput-versus-latency tradeoff behind every API price.

Jailbreak (Red-Teaming)

A jailbreak is an input that steers an AI past its safety training to produce content it was built to refuse. How they work, and how they are defended.

KV Cache

The memory an LLM keeps so it never recomputes the past — why a long context costs money, and how many users one GPU can actually serve at once.

Large Language Model (LLM)

A large language model predicts the next token of text. How that produces answers, where its knowledge is stored, and why an LLM is not a chatbot.

LLM-as-a-Judge

Using one model to grade another's output against a rubric: how it works, the biases that break it, and the human agreement rate you can actually reach.

llms.txt

A proposed Markdown file at /llms.txt that gives AI systems a clean map of a site. Google ignores it; coding agents and doc tools use it heavily.

Low-Rank Adaptation (LoRA)

LoRA freezes a model's weights and trains two thin matrices beside each one: 98,304 numbers instead of 151 million, and no added inference latency.

Machine Unlearning

Removing the influence of specific training data from an already-trained model without full retraining, driven by erasure law, copyright, and safety.

Memory Wall

Why LLM inference waits on memory rather than maths — arithmetic intensity, the roofline, and the one fact that explains HBM scarcity and the price of a token.

Model Deployment

What has to happen between a model that trains well and users calling it: replica sizing, latency budgets, cold starts, and the rollback nobody built.

Model Routing

Model routing sends each request to the cheapest capable model — a small model handles the easy majority and hard queries escalate to a frontier model.

Multimodal AI

How one model reads an image and text at once: the picture is cut into fixed patches, each patch becomes a token, and those tokens cost context.

Natural Language Processing (NLP)

NLP is the field; large language models are now the answer to most of it. What sixty years of work was for, and which parts are still unsolved.

Positional Encoding (RoPE)

Positional encoding tells a transformer where each token sits in a sequence. Why self-attention needs it, and how rotary embeddings (RoPE) work.

Prompt Caching

Repeated prompt prefixes bill at a fraction of the input price. How prefix matching works, what silently breaks it, and where the break-even sits.

Prompt Engineering

Writing the input to a fixed, already-trained language model so it reliably produces the output you want — steering the model, not retraining it.

Prompt Injection

Prompt injection is when text an AI was meant to read gets obeyed as an instruction. Direct vs indirect, why it is unfixable, and what actually contains it.

Reasoning Model

A model trained to think at length before answering. How it differs from a chat model, what the effort dial does, and the tasks where it earns its price.

Reinforcement Learning from Human Feedback (RLHF)

How RLHF works, how it compares with DPO and GRPO, and why it makes language models verbose, agreeable and confidently wrong.

Reinforcement Learning from Verifiable Rewards (RLVR)

RLVR replaces the learned reward model with a program that checks the answer. What that buys, where it stops working, and how it differs from RLHF.

Retrieval-Augmented Generation (RAG)

A technique that retrieves relevant documents at query time and feeds them into a language model's prompt, so it answers from current, external facts.

Reward Model (RM)

The scorer that stands in for a human during RLHF: how reward models are trained, why their absolute scores mean nothing, and how they break.

Scaling Laws

The empirical curves that make AI capex rational: model loss falls predictably with compute, data and parameters — and what they do not promise.

Self-Attention

The transformer operation that lets every token in a sequence weigh every other token by relevance, computed as softmax(QKᵀ/√d_k)·V.

Self-supervised Learning (SSL)

Training a model on unlabeled data by hiding part of each input and making the model predict it — the paradigm behind next-token prediction and modern LLMs.

Semantic Understanding

Whether an AI grasps meaning well enough to act correctly on novel inputs, or is pattern-matching that breaks on rephrasing — and why that line is contested.

Speculative Decoding

A small model drafts tokens and a large one verifies them in parallel — how LLM inference gets several times faster with mathematically identical output.

Structured Outputs (Constrained Decoding)

Constrained decoding masks a language model's token distribution against a JSON schema or grammar at each step, so the output is parseable by construction.

Supervised Fine-Tuning (SFT)

Supervised fine-tuning trains a pretrained language model on curated prompt-response pairs with next-token loss, teaching it to follow instructions.

Sycophancy

Sycophancy is when an AI model tells you what you want to hear — agreeing, flattering, caving when challenged — because human-preference training rewards it.

Temperature

Temperature is the number that controls how random a language model's output is — it divides the logits before softmax, from near-deterministic to diverse.

Test-Time Compute

Spending compute at inference instead of training — how reasoning models buy accuracy with thinking time, and why it moved chip demand into serving.

Text Generation

Text generation is a language model producing text one token at a time: at each step it predicts a distribution over its vocabulary, picks a token, repeats.

Token

The unit an AI model reads, writes and bills by — roughly 1.3 tokens per English word. Input, output and cached tokens are priced up to 10x apart.

Tokenization

How text becomes the integers a language model reads: BPE, WordPiece and Unigram, why " strawberry" is one token, and what that breaks.

Transformer

The 2017 neural-network architecture that processes a whole sequence in parallel using self-attention instead of recurrence — the basis of modern LLMs.

Vibe Coding

Vibe coding is building software by describing what you want to an AI and running the code it generates without reading or fully understanding it.

Zero-shot Learning

Performing a task with zero examples, on prior knowledge alone — and the split that matters: zero-shot prompting versus classical zero-shot learning.