DeepSeek-V4-Flash-GGUF

<!-- README for the Hugging Face repo `42ailab/DeepSeek-V4-Flash-GGUF`. Editing this file changes the repo landing page. -->

DeepSeek-V4-Flash · A Frontier Model That Runs Locally (284B MoE · Million-Token Context)

English | 简体中文 · Also on ModelScope · 42ailab.com

⚠️ This repository is documentation only — it holds no weight files

The two GGUF builds are 80.76 GB and 90.89 GB, both above Hugging Face's 50 GB per-file hard limit, so they cannot be hosted here. Splitting them into shards would break byte-for-byte identity with the original repo, which is exactly what makes a mirror worth verifying — so we don't. Get the weights from either of these instead:

• antirez/deepseek-v4-gguf — the original quantization, right here on Hugging Face;
• 42ailab/DeepSeek-V4-Flash-GGUF on ModelScope — our byte-for-byte mirror, for direct download inside mainland China.

The rest of this page documents the model as packaged for the 42model engine.

A 286-billion-parameter frontier model running on your own high-end computer — chat, reasoning, long-document understanding, all done locally. No cloud, free, private.

This is a mirror repo, not our original work. Attribution:

• The model itself was built by DeepSeek-AI and released under MIT (DeepSeek-V4-Flash, part of the DeepSeek-V4 family);
• The local quantization (GGUF) was produced by antirez (Salvatore Sanfilippo) using imatrix and released under MIT (he is also the author of the ds4/DwarfStar local inference engine);
• This repo (42ailab) mirrors antirez's GGUF verbatim so that it can be served from the 42model model catalog and downloaded directly inside mainland China. We did not modify the files and we did not do the quantization. If you are on Hugging Face with good connectivity, you are equally welcome to get these files from antirez's original repo — they are byte-for-byte identical.

⚠️ Frontier experimental build (Beta): to fit consumer memory this uses aggressive low-bit quantization, so output quality is below the official full-precision and hosted versions; the files are also very large (80–91 GB) and only a few high-end machines can run them. Please assess your hardware against the "Limitations" section.

1. The problem it solves

Genuinely frontier models (hundreds of billions of parameters, million-token contexts) used to run only in the cloud — you had to send your conversations and documents to someone else's servers, with the privacy concerns, connectivity requirements and costs that implies. Thanks to DeepSeek's open release and antirez's quantization, you can now run a frontier-class model fully offline on a single high-end personal computer (e.g. a 128 GB Mac plus a fast external drive):

• Feed in whole long documents: a million-token context reads an entire book or codebase in one pass before you ask anything;
• Strong reasoning and coding: DeepSeek positions this family as open models with top-tier reasoning and agentic capability;
• Sensitive data stays on the machine: legal, medical and internal-code scenarios that cannot go to the cloud stay local end to end.

2. How it works

The capability comes from DeepSeek's DeepSeek-V4-Flash — a 286-billion-parameter Mixture-of-Experts (MoE) model that activates only about 13 billion parameters per token, which is what makes it both fast and economical, with support for a million-token context. Key design points per the official material:

• Hybrid attention architecture: combining two compressed attention mechanisms to sharply reduce the compute and memory cost of very long contexts (DeepSeek reports per-token inference compute at million-token context is about 27% of the previous generation);
• More stable inter-layer signal propagation and more efficient training optimization; pre-trained on 32 trillion tokens plus two post-training stages.

Running it on a personal computer comes down to quantization: antirez used imatrix-guided (importance matrix) low-bit quantization to compress weights that would otherwise need a datacenter down to something consumer memory can hold, while preserving as much quality as possible. This repo carries his two tiers — larger is closer to the original:

• q2 tier (~81 GB): the most aggressive quantization, lowest memory requirement (96 GB+);
• q2-q4 tier (~91 GB): higher precision retained in the critical layers, better quality (128 GB+).

What 42ailab did here: mirror antirez's GGUF to ModelScope for convenient downloads inside China, and make it run with one click in the 42model engine. Neither the quantization nor the model itself is our work.

3. How well it performs

For the full public-benchmark evaluation of the model itself, refer to DeepSeek's official technical report (arXiv:2606.19348); for quantization details see antirez's original repo. Here we list only the speeds we measured on our own machine:

We deliberately distinguish peak from sustained speed: a laptop under prolonged full load throttles thermally, settling around 15 tokens/s; desktops, better cooling or AC power get closer to peak.

4. Limitations and what's next

• Experimental aggressive quantization: q2 and q2-q4 exist to fit consumer memory, so quality is below the official full-precision and hosted versions and complex tasks are more error-prone. For maximum quality, use DeepSeek's hosted service or a full-precision deployment.
• High hardware bar: q2 needs about 96 GB of memory, q2-q4 needs 128 GB; the 80–91 GB files must live on a fast external drive (not the system disk), and both downloading and running are sensitive to disk speed. 42model runs an automatic hardware check before downloading and tells you clearly if your machine does not qualify.
• macOS (Apple Silicon) only for now: Windows / Linux support will follow in later releases.
• Next: more platforms and continued speed improvements as the engine evolves; 42ailab's own quantizations are a separate track.

5. How to download and use it

This mirror is packaged for the 42model desktop app: find DeepSeek-V4-Flash (frontier tier) in the Model Library, complete the pre-download hardware check, choose the external drive to store it on, download, then click Start to chat.

Download restrictions specific to frontier models (different from ordinary models):

• The hardware check must pass before download: memory at or above the tier's requirement (96 GB+ for q2, 128 GB+ for q2-q4), a qualifying non-system external volume, adequate disk throughput, and so on. If it fails you are told up front, rather than discovering halfway through the download that it will not run.

Files and license

These files are not hosted in this repository (see the note at the top — both exceed Hugging Face's 50 GB per-file limit). They are available from antirez's original repo on Hugging Face, and from our ModelScope mirror.

Per-file sha256 values are listed on those repos' Files pages and can be verified independently; our ModelScope mirror matches antirez's original repo byte for byte.

License:

• The model itself, DeepSeek-V4-Flash, © DeepSeek-AI, MIT (Hugging Face · ModelScope · technical report);
• The GGUF quantization, © antirez (Salvatore Sanfilippo), MIT (original repo antirez/deepseek-v4-gguf).

This repo is a mirror of antirez's quantization, governed by the MIT licenses above. By using it you agree to the upstream terms.

Citation

For the model itself, please cite upstream DeepSeek (the DeepSeek-V4 technical report):

@misc{deepseekai2026deepseekv4,
      title={DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence},
      author={DeepSeek-AI},
      year={2026},
}

The GGUF quantization was produced by antirez; please also credit the original repo at https://huggingface.co/antirez/deepseek-v4-gguf.

Contact us (for mirror / engine questions only): [email protected]

About us

42ailab — an AI research lab exploring the boundaries of intelligence. Grounded in cognitive science, we work toward a deep integration of AI and human intelligence — to truly understand and augment intelligence, carbon-based and silicon-based alike.

42model — a high-performance local inference engine from 42ailab that runs translation, transcription, recognition, chat and coding on your own machine, free and private; with optional cloud compute for fine-tuning your own models and bringing them back to run locally.