Introduction
If you read a 2026 model card, you'll see numbers that look contradictory: "1.6 trillion parameters" next to "49 billion active." That's not a typo — it's Mixture of Experts (MoE), the architecture that quietly took over the field. MoE is the reason models can keep getting bigger and smarter without getting proportionally slower and more expensive to run. It's the single most important structural idea in current large language models, and it's simpler than the jargon suggests.
This explainer covers what MoE actually is, the crucial distinction between total and active parameters, why 2026 went almost entirely MoE, and the trade-offs that come with it. If you've ever wondered how a model can be "trillions of parameters" and still respond quickly, this is the answer.
The problem MoE solves
A traditional dense model runs every one of its parameters on every token. That's simple and effective, but it ties two things together that you'd rather keep separate: capacity (how much the model can know and do) and cost (how much compute each token takes). Want a smarter model? Add parameters. But now every token costs more to process, forever. Dense scaling makes the model more capable and more expensive in lockstep.
MoE breaks that link. The insight: most of what a model knows isn't needed for any single token. Processing the word "the" doesn't require the parameters that encode organic chemistry. So why run all of them every time?
How it works: experts and a router
An MoE model replaces the big feed-forward block in each transformer layer with many smaller ones — the experts — plus a small router (or gating network) that decides which experts handle each token.
For every token, the router picks a small subset — say the top 2 of 128 experts — and only those run. The other 126 sit idle for that token. Different tokens get routed to different experts, and over training the experts specialize: some drift toward code, some toward language, some toward math. The router learns to send each token to the ones most likely to help. That's the whole idea — many specialists, a dispatcher, and only a few working at a time.
Total vs active parameters: the number that matters
This is the distinction to internalize:
- Total parameters — every weight in the model. This drives capacity (and memory).
- Active parameters — the ones actually used for a given token. This drives speed and cost.
In an MoE model these diverge enormously. Real 2026 examples make it concrete:
- DeepSeek V4-Pro: ~1.6 trillion total / ~49 billion active — it has the knowledge of a 1.6T model but the per-token compute of a ~49B one.
- GLM-5.2: a 753B-parameter MoE.
- Kimi K3: a 2.8-trillion-parameter MoE.
A dense 1.6T model would be ruinously slow to run. DeepSeek V4-Pro delivers comparable capacity while only paying for ~49B parameters per token. That's the MoE bargain in one line: frontier capacity, fractional compute.
Why 2026 went all-in on MoE
By mid-2026, MoE had become the dominant design pattern across essentially every serious open-weight release — DeepSeek, GLM, Kimi, Qwen, MiniMax, and more all ship MoE flagships. The reasons follow directly from the mechanism:
- Better cost-performance. More capability per dollar of inference, which is decisive when you're serving millions of requests. (See NVIDIA's work on MoE inference efficiency.)
- Faster inference than a dense model of equivalent total size, because only a slice runs per token.
- Stronger specialization, as experts carve up the problem space during training.
- Scaling headroom. You can grow total capacity by adding experts without proportionally growing the per-token compute — a friendlier curve than dense scaling laws.
Put simply, MoE is how the field kept scaling capability after dense scaling got too expensive to sustain.
The trade-offs
MoE isn't free. Three real costs come with it:
- Memory, not compute, becomes the wall. Because any expert might be chosen, all of them must be loaded into memory. So a large MoE is cheap per token but still needs enough VRAM to hold every parameter — which is why a trillion-parameter MoE can be fast yet still demand multiple high-memory GPUs. Active parameters set the speed; total parameters set the memory bill.
- Serving complexity. Routing, load-balancing across experts, and keeping GPUs busy are genuinely harder than serving a dense model. This is active engineering territory.
- Quantization is trickier. Compressing an MoE well — especially the experts — is an ongoing research area, because naive quantization can hurt the specialized experts unevenly.
What it means for you
If you're choosing or running models, MoE changes how you read the specs. Look at active parameters for speed and cost, and total parameters for capability and hardware. A model advertised at "744B" might run faster than a dense "70B" if its active count is low — but it'll need far more memory to load. When you plan self-hosting hardware, size for total parameters; when you estimate latency and price, think about active ones. This is exactly the reasoning behind picking among open-weight models, where a smaller active count is what makes a giant model practical to run.
Conclusion
Mixture of Experts is the architecture that let models keep growing after dense scaling hit an economic ceiling. By splitting the network into specialized experts and activating only a few per token, MoE decouples capacity from cost — trillions of parameters' worth of knowledge at a fraction of the per-token compute. The trade is memory and serving complexity, but the win was decisive enough that by 2026 nearly every frontier open-weight model is MoE. Next time a model card shows "2.8T total / a few billion active," you'll know exactly what it's telling you. For the fundamentals, see our glossary on Mixture of Experts and parameters.