---
source: 'https://howaiworks.ai/blog/alibaba-qwen-3-8-flash-next-local-gguf'
section: blog
title: Qwen3.8-Flash-Next Runs Locally in 75 GB of RAM
description: >-
  Alibaba's 125B multimodal MoE is out as Unsloth GGUFs. The 1-bit build fits in
  about 75 GB of RAM or unified memory, with no GPU VRAM required.
date: '2026-08-27'
author: HowAIWorks Team
tags:
  - Alibaba
  - Qwen
  - MoE
  - Quantization
  - Open Weights
  - Multimodal AI
  - Local LLM
  - Inference
  - AI News
readingTime: 5 minutes
newsSource: Alibaba
breakingNews: false
isReviewed: false
---

# Qwen3.8-Flash-Next Runs Locally in 75 GB of RAM

> Alibaba's 125B multimodal MoE is out as Unsloth GGUFs. The 1-bit build fits in about 75 GB of RAM or unified memory, with no GPU VRAM required.

## Introduction

Alibaba's Qwen team released **Qwen3.8-Flash-Next** on August 26, 2026 — a 125B-parameter multimodal [Mixture-of-Experts](https://howaiworks.ai/glossary/mixture-of-experts) model billed as an early preview of the Qwen4 architecture. Within a day, Unsloth published [quantized](https://howaiworks.ai/glossary/quantization) GGUF builds and a run guide, and the headline number is the one that matters to anyone without a datacenter: the smallest build fits in roughly **75 GB of RAM or unified memory**, with no GPU VRAM required.

That is a 125B multimodal model on a well-specced laptop. Below is why it works, what hardware actually clears the bar, and what the quantization costs you.

## The architecture behind the number

Qwen3.8-Flash-Next is three components, not one:

- A **125B MoE backbone** — 512 experts, 10 routed plus 1 shared, with only **6B parameters active per token**
- A **51B N-gram embedding table**: 20 million bigram and trigram entries at layer 2, adding capacity through deterministic lookups rather than computation, and designed to sit in system RAM rather than GPU memory
- A **4B multi-token prediction module**

Attention is hybrid — Gated DeltaNet in three of every four layers, Qwen Sparse Attention in the fourth, across 48 layers with a 4-branch gated residual. The [context window](https://howaiworks.ai/glossary/context-window) is 262,144 tokens natively, extensible to 1M with YaRN. The model is [multimodal](https://howaiworks.ai/glossary/multimodal-ai), with selectable reasoning effort: xhigh, medium, low or none.

The 6B active-parameter figure is what makes local [inference](https://howaiworks.ai/glossary/inference) viable. Generating a token touches a small fraction of the weights, so per-token bandwidth demand falls far below what 125B parameters implies — and bandwidth, not compute, is what usually makes CPU inference painful. Sparse activation sidesteps most of the [memory wall](https://howaiworks.ai/glossary/memory-wall), which is why running from system RAM lands closer to VRAM speeds here than it would for a dense model of similar size.

Alibaba also reports the model beating Qwen3.7-Plus at roughly **one-ninth the training cost** — the efficiency story runs through training as well as inference.

One trade-off Qwen's materials state plainly: **sparse activation cuts compute, not storage.** At FP8 the model still occupies 172.78 GiB. Quantization, not sparsity, is what gets it onto consumer hardware.

## What fits in what

Unsloth's dynamic GGUF builds:

| Quantization | Size on disk | RAM / unified memory |
|---|---|---|
| 1-bit (UD-IQ1_S) | 72.5 GB | ~75 GB |
| 2-bit (UD-Q2_K_XL) | 78.9 GB | ~79 GB |
| 3-bit (UD-Q3_K_XL) | 90 GB | ~90 GB |
| 4-bit (UD-IQ4_XS) | 93.7 GB | ~112 GB |
| BF16 (unquantized) | 355 GB | 355 GB |

Mapped onto real machines:

- **128 GB unified-memory Mac** (M4/M5 Max, Mac Studio) — the 4-bit build at the full 262K context
- **96 GB GPU** (RTX PRO 6000) — the 2-bit build entirely in VRAM, or 4-bit with experts split out to system RAM
- **24 GB GPU** (RTX 3090/4090/5090) plus 96 GB+ system RAM — the 4-bit build with experts held in system RAM
- **No GPU, ~80 GB RAM** — the 1-bit builds

Context is cheaper than usual here: because the sparse-attention layers keep only 2 KV heads, the entire 262K window costs about **6.5 GB of cache** — a fraction of what a conventional [attention](https://howaiworks.ai/glossary/attention-mechanism) stack would need at that length.

The 1-bit build is 79% smaller than BF16, keeps the N-gram and PLE components at 4-bit, and retains 80% top-1 accuracy per Unsloth. Treat that as a vendor figure on the vendor's own evaluation: useful evidence the build is not broken, not a substitute for testing your workload. The 2-bit build costs 4 GB more and is the safer default.

Simon Willison ran both on a DGX Spark and reported good results from UD-Q2_K_XL at xhigh reasoning effort. **No published tokens-per-second figures exist yet** — not in Unsloth's docs, not in the community writeups. The memory question is answered; the speed question is not.

## The benchmark claim, checked

Qwen's model card reports Qwen3.8-Flash-Next beating **Claude Opus 4.6 at max reasoning effort** on LiveCodeBench v6 (91.9), SWE-bench Multilingual (81.0), SWE-bench Pro (62.5), DeepSWE 1.1 (58.7), AndroidWorld (84.5), RealWorldQA (88.5) and MathVision with a code interpreter (95.7). The base model tops 8 of 14 benchmarks including MMLU-Pro, SuperGPQA, BBH and GSM8K.

Two qualifications belong next to those numbers. The lead is not universal — Opus 4.6 wins Humanity's Last Exam 40.0 to 35.9, and DeepSeek-V4-Flash-0731 leads NL2Repo-Bench at 54.2. More importantly for anyone reading "beats Opus" as the summary: **Opus 4.6 is not Anthropic's current flagship.** [Claude Opus 5](https://howaiworks.ai/models/claude-opus) shipped on July 24, 2026, with Claude Fable 5 above the Opus tier. The comparison is two releases back.

Everything here is self-reported by the developer, and benchmark scores and real-world behavior routinely diverge. The claim worth attention is not the leaderboard position — it is that scores in this neighborhood now come out of something you can run yourself.

## Before you download

- **License**: `qwen-community-1.0`, not Apache-2.0. Read the terms before commercial deployment.
- **Tooling**: llama.cpp needs PR #27742 for correct MoE support. Unsloth Desktop bundles a working build and tunes parameters automatically; `unsloth run` starts a llama-server-backed API. [Ollama](https://howaiworks.ai/ai-tools/ollama) and [LM Studio](https://howaiworks.ai/ai-tools/lm-studio) need the same llama.cpp change first.
- **Sampling settings**: thinking mode wants temperature 1.0, top_p 0.95, top_k 20, presence_penalty 0.0; instruct mode wants 0.7, 0.80, 20 and 1.5. The gap is wide enough that one set used for both modes will visibly degrade output.
- **The API alternative**: the production Qwen3.8-Flash is priced at $0.16 per million input tokens and $0.47 per million output tokens on QwenCloud — roughly a twelfth of [Qwen's flagship tier](https://howaiworks.ai/models/qwen). Local inference here buys privacy and control, not savings.

## Conclusion

The notable thing about Qwen3.8-Flash-Next is not that it wins benchmarks against a model Anthropic has since superseded twice. It is that a 125B multimodal MoE with a 262K context window runs on 75 GB of ordinary RAM, without a GPU, a day after release. Sparse activation made the bandwidth affordable, aggressive quantization made the footprint affordable, and the combination moved a class of model that needed multi-GPU infrastructure onto hardware people already own.

The caveats are real: a community license rather than Apache-2.0, an unmerged llama.cpp PR, vendor-reported accuracy retention at 1-bit, and no throughput numbers at all. The second week of community testing — particularly on how the 1-bit build holds up and how fast any of this actually generates — is what will settle whether the headline survives contact with use.

## Sources

- [Qwen/Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) — official model card, benchmark table and license
- [Qwen3.8-Flash-Next: How to Run Locally](https://unsloth.ai/docs/models/qwen3.8-next) — Unsloth documentation
- [unsloth/Qwen3.8-Flash-Next-GGUF](https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF) — quantization builds and sizes
- [Qwen3.8-Flash-Next](https://simonwillison.net/2026/Aug/26/qwen38-flash-next/) — Simon Willison's hands-on notes
- [Alibaba releases Qwen3.8-Flash-Next, targeting "ultimate cost efficiency"](https://the-decoder.com/alibaba-releases-qwen3-8-flash-next-targeting-ultimate-cost-efficiency/) — pricing and training-cost reporting
- [Alibaba's Qwen Team Releases Qwen3.8-Flash-Next](https://www.marktechpost.com/2026/08/26/alibabas-qwen-team-releases-qwen3-8-flash-next-a-125b-multimodal-moe-with-6b-active-parameters-previewing-the-qwen4-architecture/) — architecture breakdown

## Frequently Asked Questions

### How much RAM do you need to run Qwen3.8-Flash-Next locally?

Unsloth's 1-bit UD-IQ1_S build is 72.5 GB on disk and needs roughly 75 GB of RAM or unified memory. The 2-bit build needs about 79 GB, and the 4-bit UD-IQ4_XS build wants around 112 GB once you leave headroom for context.

### What hardware can actually run it?

A 128 GB unified-memory Mac such as an M4 or M5 Max runs the 4-bit build at the full 262K context. A 96 GB GPU runs the 2-bit build entirely in VRAM. A 24 GB card paired with 96 GB or more of system RAM runs the 4-bit build with the expert weights held in system RAM.

### Does Qwen3.8-Flash-Next need a GPU?

No. It is designed to run from CPU system RAM or unified memory. Only 6B of its 125B parameters activate per token, so per-token memory bandwidth demand is low enough that CPU or unified-memory inference stays closer to GPU speeds than is typical for a model this size.

### Is Qwen3.8-Flash-Next open source?

It is open weight but not Apache-2.0. The weights ship under the qwen-community-1.0 license, so check the terms before using the model commercially.

### Does Qwen3.8-Flash-Next really beat Claude Opus 4.6?

Qwen reports wins over Claude Opus 4.6 at max reasoning effort on several coding, agentic and multimodal benchmarks, but not all of them — Opus 4.6 still leads on Humanity's Last Exam, 40.0 to 35.9. Opus 4.6 is also not Anthropic's current flagship; Claude Opus 5 shipped in July 2026.

### What does the 51B N-gram embedding table do?

It is a 20-million-entry bigram and trigram lookup table at layer 2. It adds capacity through deterministic lookups rather than computation, and it is designed to live in system RAM rather than GPU memory.

---

Source: https://howaiworks.ai/blog/alibaba-qwen-3-8-flash-next-local-gguf — HowAIWorks.ai