deepseek-v4-flash

hipfire DeepSeek V4 Flash (mq2lloyd)

A mixed-precision packaging of deepseek-ai/DeepSeek-V4-Flash (284 B total / 13 B active per upstream) for the hipfire Rust-native inference engine on AMD RDNA GPUs.

Upstream is shipped in FP8 (e4m3) with FP4 routed-expert weights. This packaging rewrites the dominant weight class — the 256 routed MoE experts per layer — as 2-bit MagnumQuant-Lloyd (MQ2-Lloyd), and keeps everything else as Q8F16 or F16. The container suffix .mq2lloyd names that dominant class, but the file is not a uniform 2-bit dump.

The file format is hipfire's HFQ container and is not GGUF / safetensors / AWQ compatible — it only loads in hipfire.

What's inside (verified by enumerating the file's tensor table)

The 86.2 GB file contains 34 223 tensors. By hipfire QuantType enum:

  33024  qt=19 (MQ2G256Lloyd)        – 256 routed experts × 3 (w1/w2/w3) × 43 layers
    389  qt=3  (Q8F16)               – shared experts, main attn, embed, lm_head, router gates
    807  qt=1  (F16)                 – norms, compressor, indexer, hc_*, attn_sink
      3  qt=22 (TidI32)              – hash-router fast-path (3 layers only)

The runtime SWA K and V state (window = 128) is F32 and is allocated live at session start — it is not persisted in the file.

Files

DSpark speculative drafter (-dspark sidecar)

deepseek-v4-flash-dspark.mq2lloyd is an optional multi-token draft module that replaces the single in-trunk MTP head with a 3-stage chain (mtp.0 → mtp.1 → mtp.2) proposing a block of 5 tokens per step. It is additive to the main file and to the MTP companion — attach whichever you want; DSpark auto-wins over MTP when both are present.

Verified by enumerating the file's tensor table (hfq_dump): 2 376 tensors, arch_id = 9, same mixed-precision scheme as the main file:

  2304  qt=19 (MQ2G256Lloyd)   – 256 routed experts × 3 (w1/w2/w3) × 3 draft stages
    31  qt=3  (Q8F16)          – per-stage attention, shared experts, main_proj, markov heads, confidence head
    41  qt=1  (F16)            – norms, mHC gating (hc_head_*), main_norm

The three stages are near-symmetric full DeepSeek-V4 blocks (791 / 789 / 796 tensors) with role-specific heads:

• mtp.0 — owns the embed/main-hidden entry: main_proj (Linear 12288→4096, no bias) + main_norm (RMS). Draft input comes from main_proj + a noise block (dspark_noise_token_id = 128799), not the MTP head's e_proj/h_proj mixing.
• mtp.1 — middle draft block.
• mtp.2 — owns the output heads: hc_head_{base,fn,scale}, markov_head.{markov_w1, markov_w2} (Embed 129280→256 / Linear 256→129280) for the Markov token-bias prior, and confidence_head.proj (Linear 4352→1, fp32) driving confidence-truncated acceptance.

Config (from the file's metadata blob): dspark_block_size = 5, dspark_target_layer_ids = [40, 41, 42], dspark_markov_rank = 256, dspark_noise_token_id = 128799.

Enable it with hipfire run deepseek-v4-flash --spec dspark (force) or --spec auto (DSpark preferred when the sidecar is present). Acceptance can be tuned with --dspark-conf-threshold <0..1> (deepseek4 default 0.3).

Loading

This model is registered in hipfire's CLI registry under deepseek-v4-flash (aliases: deepseek4, deepseek-v4). The registry entry pulls the main file together with both the MTP and DSpark companions.

# 1. Install hipfire (one-time):
curl -L https://raw.githubusercontent.com/Kaden-Schutt/hipfire/master/scripts/install.sh | bash

# 2. Pull the model (downloads both .mq2lloyd files into ~/.hipfire/models/):
hipfire pull deepseek-v4-flash

# 3. Run it:
hipfire run deepseek-v4-flash "Write a fibonacci function in C"
#    or interactive chat:
hipfire run deepseek-v4-flash
#    or expose an OpenAI-compatible HTTP server on :11435:
hipfire serve

hipfire run auto-pulls if the files aren't present, so step 2 is optional. Both HIPFIRE_DEEPSEEK4_MOE and HIPFIRE_DEEPSEEK4_UPLOAD_EXPERTS default to on at the engine level — no env vars are required.

From source (development path)

git clone https://github.com/Kaden-Schutt/hipfire
cd hipfire
cargo build --release -p hipfire-arch-deepseek4 --example deepseek4_chat

# Pull the files (uses `hf` CLI or hipfire's registry-driven pull above):
hf download nwoolmer/hipfire-deepseek-v4-flash \
    deepseek-v4-flash.mq2lloyd deepseek-v4-flash-mtp.mq2lloyd \
    deepseek-v4-flash-dspark.mq2lloyd \
    --local-dir ~/.hipfire/models/

# Direct chat binary (DSML chat template, EOS stop, multi-turn KV):
HIPFIRE_DEEPSEEK4_MODEL=~/.hipfire/models/deepseek-v4-flash.mq2lloyd \
    ./target/release/examples/deepseek4_chat

For programmatic access, crates/hipfire-runtime/examples/daemon.rs exposes the engine as a JSON-lines IPC service over stdin/stdout (spawned by the Bun CLI front-end); it dispatches DeepSeek V4 Flash via arch_id = 9.

Architecture

The packaging follows the V4F architecture as described in the DeepSeek V4 paper. Every value below is read from the config JSON embedded in the HFQ file's metadata blob:

Tensor-presence cross-check against the file (also verified above):

• Layers 0 and 1: dense attention (wq_a/b, wkv, wo_a/b only — no compressor / no indexer).
• Layers 2–42: compressed-KV attention (every layer has attn.compressor.{wkv,wgate,ape,norm}).
• Even layers 2, 4, …, 42 (21 layers): also carry the indexer block (attn.indexer.{wq_b, weights_proj, compressor.*}).
• Layers 0, 1, 2: also ship the ffn.gate.tid2eid hash-router fast-path table.
• The MTP companion file contains a single layer (mtp.0.*) with its own attention block, 256 routed experts, a shared expert, a full mHC stack (hc_attn_*, hc_ffn_*, hc_head_*), and input-side projection/norm tensors (e_proj, h_proj, enorm, hnorm) used by the MTP head's input mixing.

Performance

Measured 2026-05-28 on AMD Radeon 8060S (gfx1151, Strix Halo APU, 128 GB UMA), hipfire at tag v0.2.0 (commit 3d456e5c), ROCm 7.2.1, SWA attention path, temp=0.7 top_k=40, prompt_normalize on.

Cold-process load — weight upload from both HFQ files — is ~44 s on the 8060S (measured by an inline timestamping wrapper from process start until the engine prints DeepSeek V4 ready.). Plain decode is DRAM-bandwidth-bound; faster memory (RDNA3 desktop GDDR6, future RDNA4) scales decode roughly proportionally.

Compatibility

• GPU: AMD RDNA3 / RDNA3.5 with HIP + WMMA. Validated on gfx1151 (Radeon 8060S, Strix Halo) for this build of the V4F weights. The engine has gfx1100 (RX 7900-class) kernels in tree but they were not exercised against this specific file; RDNA1/2 (gfx1010 / gfx103x) and gfx12 are tracked targets of the broader hipfire project but not recommended for running this model.
• OS: Linux with amdgpu kernel driver. Built and measured against ROCm 7.2.1 / HIP 7.2.
• Memory: 86.2 GB for the main file + ~2 GB for the MTP companion + ~6 GB for the optional DSpark drafter + several GB working set during decode. Strix Halo class systems with 128 GB UMA are the comfortable target; discrete-GPU configs need ~96 GB+ of VRAM.
• Tested context length: end-to-end inference was exercised at prompts up to ~1.2 k tokens and generations up to 256 tokens. Upstream's stated max position (1 048 576 via YaRN) was not exercised here.

License

The upstream model (deepseek-ai/DeepSeek-V4-Flash) is MIT-licensed; the weights in this packaging inherit those terms.

The hipfire engine that produced and consumes this format is dual-licensed MIT / Apache-2.0 at the user's option (see LICENSE-MIT, LICENSE-APACHE, and NOTICE in the engine repo).

Acknowledgements

• DeepSeek AI — original DeepSeek V4 Flash weights and architecture.
• Salvatore Sanfilippo (antirez) — DwarfStar, a focused C/Metal/CUDA/ROCm reference engine for DeepSeek V4 Flash. The hipfire impl cross-validates against it for MTP wiring, HC reduction, and KV layout.

Citation

If you use this packaging, please cite the upstream model, the hipfire engine (per its CITATION.cff), and this HF release:

@misc{deepseekai2026deepseekv4,
  title  = {DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence},
  author = {DeepSeek-AI},
  year   = {2026},
  url    = {https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash}
}

@software{hipfire,
  title   = {hipfire — Rust-native LLM inference for AMD RDNA / CDNA},
  author  = {Schutt, Kaden},
  year    = {2026},
  version = {0.2.0},
  url     = {https://github.com/Kaden-Schutt/hipfire}
}

@misc{hipfire-deepseek-v4-flash,
  title  = {hipfire DeepSeek V4 Flash (MQ2-Lloyd)},
  author = {Woolmer, Nick},
  year   = {2026},
  url    = {https://huggingface.co/nwoolmer/hipfire-deepseek-v4-flash}
}