Qwen3.8-Flash-Next Runs Locally in 75 GB of RAM

Introduction

Alibaba's Qwen team released Qwen3.8-Flash-Next on August 26, 2026 — a 125B-parameter multimodal Mixture-of-Experts model billed as an early preview of the Qwen4 architecture. Within a day, Unsloth published quantized GGUF builds and a run guide, and the headline number is the one that matters to anyone without a datacenter: the smallest build fits in roughly 75 GB of RAM or unified memory, with no GPU VRAM required.

That is a 125B multimodal model on a well-specced laptop. Below is why it works, what hardware actually clears the bar, and what the quantization costs you.

The architecture behind the number

Qwen3.8-Flash-Next is three components, not one:

• A 125B MoE backbone — 512 experts, 10 routed plus 1 shared, with only 6B parameters active per token
• A 51B N-gram embedding table: 20 million bigram and trigram entries at layer 2, adding capacity through deterministic lookups rather than computation, and designed to sit in system RAM rather than GPU memory
• A 4B multi-token prediction module

Attention is hybrid — Gated DeltaNet in three of every four layers, Qwen Sparse Attention in the fourth, across 48 layers with a 4-branch gated residual. The context window is 262,144 tokens natively, extensible to 1M with YaRN. The model is multimodal, with selectable reasoning effort: xhigh, medium, low or none.

The 6B active-parameter figure is what makes local inference viable. Generating a token touches a small fraction of the weights, so per-token bandwidth demand falls far below what 125B parameters implies — and bandwidth, not compute, is what usually makes CPU inference painful. Sparse activation sidesteps most of the memory wall, which is why running from system RAM lands closer to VRAM speeds here than it would for a dense model of similar size.

Alibaba also reports the model beating Qwen3.7-Plus at roughly one-ninth the training cost — the efficiency story runs through training as well as inference.

One trade-off Qwen's materials state plainly: sparse activation cuts compute, not storage. At FP8 the model still occupies 172.78 GiB. Quantization, not sparsity, is what gets it onto consumer hardware.

What fits in what

Unsloth's dynamic GGUF builds:

Mapped onto real machines:

• 128 GB unified-memory Mac (M4/M5 Max, Mac Studio) — the 4-bit build at the full 262K context
• 96 GB GPU (RTX PRO 6000) — the 2-bit build entirely in VRAM, or 4-bit with experts split out to system RAM
• 24 GB GPU (RTX 3090/4090/5090) plus 96 GB+ system RAM — the 4-bit build with experts held in system RAM
• No GPU, ~80 GB RAM — the 1-bit builds

Context is cheaper than usual here: because the sparse-attention layers keep only 2 KV heads, the entire 262K window costs about 6.5 GB of cache — a fraction of what a conventional attention stack would need at that length.

The 1-bit build is 79% smaller than BF16, keeps the N-gram and PLE components at 4-bit, and retains 80% top-1 accuracy per Unsloth. Treat that as a vendor figure on the vendor's own evaluation: useful evidence the build is not broken, not a substitute for testing your workload. The 2-bit build costs 4 GB more and is the safer default.

Simon Willison ran both on a DGX Spark and reported good results from UD-Q2_K_XL at xhigh reasoning effort. No published tokens-per-second figures exist yet — not in Unsloth's docs, not in the community writeups. The memory question is answered; the speed question is not.

The benchmark claim, checked

Qwen's model card reports Qwen3.8-Flash-Next beating Claude Opus 4.6 at max reasoning effort on LiveCodeBench v6 (91.9), SWE-bench Multilingual (81.0), SWE-bench Pro (62.5), DeepSWE 1.1 (58.7), AndroidWorld (84.5), RealWorldQA (88.5) and MathVision with a code interpreter (95.7). The base model tops 8 of 14 benchmarks including MMLU-Pro, SuperGPQA, BBH and GSM8K.

Two qualifications belong next to those numbers. The lead is not universal — Opus 4.6 wins Humanity's Last Exam 40.0 to 35.9, and DeepSeek-V4-Flash-0731 leads NL2Repo-Bench at 54.2. More importantly for anyone reading "beats Opus" as the summary: Opus 4.6 is not Anthropic's current flagship. Claude Opus 5 shipped on July 24, 2026, with Claude Fable 5 above the Opus tier. The comparison is two releases back.

Everything here is self-reported by the developer, and benchmark scores and real-world behavior routinely diverge. The claim worth attention is not the leaderboard position — it is that scores in this neighborhood now come out of something you can run yourself.

Before you download

• License: qwen-community-1.0, not Apache-2.0. Read the terms before commercial deployment.
• Tooling: llama.cpp needs PR #27742 for correct MoE support. Unsloth Desktop bundles a working build and tunes parameters automatically; unsloth run starts a llama-server-backed API. Ollama and LM Studio need the same llama.cpp change first.
• Sampling settings: thinking mode wants temperature 1.0, top_p 0.95, top_k 20, presence_penalty 0.0; instruct mode wants 0.7, 0.80, 20 and 1.5. The gap is wide enough that one set used for both modes will visibly degrade output.
• The API alternative: the production Qwen3.8-Flash is priced at $0.16 per million input tokens and $0.47 per million output tokens on QwenCloud — roughly a twelfth of Qwen's flagship tier. Local inference here buys privacy and control, not savings.

Conclusion

The notable thing about Qwen3.8-Flash-Next is not that it wins benchmarks against a model Anthropic has since superseded twice. It is that a 125B multimodal MoE with a 262K context window runs on 75 GB of ordinary RAM, without a GPU, a day after release. Sparse activation made the bandwidth affordable, aggressive quantization made the footprint affordable, and the combination moved a class of model that needed multi-GPU infrastructure onto hardware people already own.

The caveats are real: a community license rather than Apache-2.0, an unmerged llama.cpp PR, vendor-reported accuracy retention at 1-bit, and no throughput numbers at all. The second week of community testing — particularly on how the 1-bit build holds up and how fast any of this actually generates — is what will settle whether the headline survives contact with use.

Sources

• Qwen/Qwen3.8-Flash-Next — official model card, benchmark table and license
• Qwen3.8-Flash-Next: How to Run Locally — Unsloth documentation
• unsloth/Qwen3.8-Flash-Next-GGUF — quantization builds and sizes
• Qwen3.8-Flash-Next — Simon Willison's hands-on notes
• Alibaba releases Qwen3.8-Flash-Next, targeting "ultimate cost efficiency" — pricing and training-cost reporting
• Alibaba's Qwen Team Releases Qwen3.8-Flash-Next — architecture breakdown