Downloads · 30 days
68
9% of all-time downloads
bluehawana/DeepSeek-V4-Flash-0731-IQ2_M-GGUF
DeepSeek-V4-Flash-0731-IQ2_M-GGUF is a machine learning model from bluehawana. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Single-file IQ2M quant of DeepSeek-V4-Flash-0731 (284B MoE, 13B active per token, MIT) merged from AtomicChat's sharded GGUFs so Ollama can pull it. ~33 tok/s on a 128 GB Apple Silicon Mac. Model by DeepSeek, quantiza…
Downloads · 30 days
68
9% of all-time downloads
All-time downloads
727
Public
Repo size
104 GB
Likes
0
Public
Click a slice to open those files.
.gguf104 GB · 100%
From the Hugging Face model README
Single-file IQ2_M quant of DeepSeek-V4-Flash-0731 (284B MoE, 13B active per token, MIT) merged from AtomicChat's sharded GGUFs so Ollama can pull it. ~33 tok/s on a 128 GB Apple Silicon Mac. Model by DeepSeek, quantization by AtomicChat — we only merged and documented.
Run it on a Mac with one command — no cloning, no merging:
ollama run hf.co/bluehawana/DeepSeek-V4-Flash-0731-IQ2_M-GGUF
Or from the Ollama registry:
ollama run bluehawana/deepseek-v4-flash:iq2_m
~104 GB download. Tested on a MacBook Pro M5 Max (128 GB unified memory): loads in ~25 s.
The excellent AtomicChat quants
are published as sharded GGUFs, which Ollama can't pull from the registry —
and (as of Ollama 0.32.12) ollama create fails on them locally too
(split GGUF ... has 1 shards, expected 4). This repo is the same IQ2_M quant
merged into a single file with llama.cpp's llama-gguf-split --merge, so
ollama run hf.co/... just works.
Scripts, the do-it-yourself recipe (works for any sharded GGUF, any quant), and Mac-specific tips: https://github.com/bluehawana/ollama-deepseek-v4-flash-iq2_m-mac-m5max
| Run | Prompt tok | Gen tok | Prefill tok/s | Gen tok/s | Total |
|---|---|---|---|---|---|
| Short answer | 15 | 128 | 51.9 | 33.9 | 4.2 s |
| Short story | 18 | 256 | 59.4 | 33.6 | 8.0 s |
| Essay | 25 | 512 | 64.8 | 33.3 | 15.8 s |
| Long-prompt summary | 2,774 | 64 | 544.9 | 32.4 | 7.2 s |
Measured via Ollama's /api/generate (its own eval_count/eval_duration
fields), one pass per row, reasoning tokens included. These speed numbers are
our own measurements of this merged file; all quality metrics cited in this
card are AtomicChat's.
<think>…</think> before answers./set parameter num_ctx 8192) —
103 GB of weights leaves little KV-cache headroom.We did not train or quantize this model. This repo's only contribution is
merging AtomicChat's four GGUF shards into one file (llama.cpp
llama-gguf-split --merge, a byte-preserving operation) so Ollama can pull it,
plus Mac-specific documentation. All quality metrics quoted anywhere in this
project (perplexity, KL divergence, 83.6% top-1 match) are AtomicChat's own
measurements from their model card.