Term tag: Edge AI

Autonomous Systems

Automation runs a procedure someone wrote down; autonomy picks the procedure at run time. Sense-plan-act loops, operational design domains and autonomy levels.

Edge AI

Edge AI runs the model on the device that captured the data instead of in the cloud — what that buys in latency, bandwidth and privacy, and what it costs.

Federated Learning

Federated learning trains a shared model across many devices without moving their raw data: the local-train, send-updates, aggregate loop.

Inference

Running a trained model to get an answer. Training is paid once; inference is paid on every request — and it is where most AI money goes.

Knowledge Distillation

How a small model learns what a big one knows: the teacher's full probability distribution carries structure the hard labels do not — Hinton's dark knowledge.

Model Compression

Making a trained model smaller and cheaper to run — quantization, pruning, distillation and low-rank methods, and which one your problem actually needs.

Model Deployment

What has to happen between a model that trains well and users calling it: replica sizing, latency budgets, cold starts, and the rollback nobody built.

Neural Processing Unit (NPU)

An NPU is a fixed-function matrix engine inside a phone or laptop chip that runs AI models at a few watts — and why TOPS is the wrong number to…

Quantization

Running a model in fewer bits — FP16, FP8, INT4 — to cut memory and cost, what accuracy it actually costs, and why a memory-bound model also gets faster.

Small Language Model (SLM)

A language model small enough to run on the device that uses it — roughly under 5B parameters. Trained far past compute-optimal to buy cheap inference.