Term tag: GPU

AI Data Center

Why AI facilities are measured in megawatts rather than servers — power density, liquid cooling, and the grid connection that has become the real constraint.

AI Infrastructure

Why AI clusters are designed around memory bandwidth and interconnect rather than FLOPs, and what that changes when you size a training run or a serving fleet.

Cloud Computing

Cloud computing is the on-demand delivery of computing power, database storage, applications, and other IT resources via the internet with pay-as-you-go.

CUDA

NVIDIA's software layer for GPUs — the programming model that turned graphics cards into AI hardware, and the real reason rivals with comparable silicon lose.

Disaggregated Serving (Prefill/Decode)

Disaggregated serving runs LLM prefill and decode on separate GPU pools because the two phases hit opposite limits: compute versus memory bandwidth.

Distributed Training

How one model is trained across thousands of GPUs — data, tensor and pipeline parallelism, and why the network rather than the chip becomes the bottleneck.

FlashAttention

The exact-attention algorithm that made long context affordable: it never writes the n x n score matrix to GPU memory, so it runs faster and changes no output.

FLOPs (Floating Point Operations)

The unit AI compute is measured in — how a training run's FLOP budget is counted, why FLOP and FLOPS are not the same, and why regulators use the number.

GPU Computing

Why a graphics chip runs AI: a GPU spends its transistors on arithmetic instead of on making one thread fast — and what that trade wins and costs.

Inference Optimization

How a served LLM is made cheap: continuous batching, the prefill/decode split, and the throughput-versus-latency tradeoff behind every API price.

Interconnect (NVLink, InfiniBand)

The wiring that moves data between AI chips: on-package memory, NVLink inside a server, InfiniBand between them — and why each tier is far slower than the last.

KV Cache

The memory an LLM keeps so it never recomputes the past — why a long context costs money, and how many users one GPU can actually serve at once.

Matrix Multiplication (GEMM)

The one operation that consumes most of an AI model's compute — and the reason AI chips are built the way they are, from tensor cores to systolic arrays.

Model Compression

Making a trained model smaller and cheaper to run — quantization, pruning, distillation and low-rank methods, and which one your problem actually needs.

Model Deployment

What has to happen between a model that trains well and users calling it: replica sizing, latency budgets, cold starts, and the rollback nobody built.

NVIDIA GPU for AI

Why AI training runs on NVIDIA rather than cheaper silicon — tensor cores, NVLink, and the software that decides how much of the datasheet you actually get.

Parallel Processing

Running many computations at the same instant across many processing units. In AI it makes matrix multiplication fast, which is why GPUs train models.

Quantization

Running a model in fewer bits — FP16, FP8, INT4 — to cut memory and cost, what accuracy it actually costs, and why a memory-bound model also gets faster.

Vectorization

Replacing element-by-element loops with operations on whole arrays at once, so the work runs as SIMD instructions on a CPU or in parallel on a GPU.