DeepSeek-V4-Flash-Vision-Strix-Halo-GGUF

DeepSeek-V4-Flash-Vision-Exp — Strix Halo ROCmFPx GGUF

[!WARNING] This model is designed to run only on AMD Strix Halo (gfx1151) using the Ember inference engine.

The importance matrices are a separate matter — they are plain llama.cpp .dat files and work with any quantizer. See Importance matrix.

An abliterated, importance-matrix-calibrated quantization of deepseek-ai/DeepSeek-V4-Flash-Vision-Exp at 2.58 BPW, with its vision tower and a DSpark drafter. Same recipe as the text-only DeepSeek-V4-Flash-0731 release, which this supersedes as Ember's default download; that one is neither deprecated nor deleted.

Two things are different here: images work, and the importance matrix was collected on images rather than text.

Artifacts

Verify any local copy against the SHA-256 values below.

Target + tower + drafter is 103,377,656,192 bytes (96.28 GiB) resident. The vision tower ships at F16 and is not quantized, because the upstream FP8 release excludes the vision weights.

Images

The tower is required for image input, and the format is PNG only. Ember's native vision path accepts non-interlaced RGB/RGBA8 PNG; JPEG, WebP and GIF are rejected rather than handed to a permissive decoder.

ember-dflash \
  -m DeepSeek-V4-Flash-Vision-Exp-Abliterated-ROCMFPx-Strix-Lean-2.58bpw.gguf \
  --vision-mmproj mmproj-DeepSeek-V4-Flash-Vision-Exp-F16.gguf \
  --ds4-expert-top-k 4

Image requests take one full-model layer-major prefill; speculative decode and resident batching are declined for them rather than silently degraded.

Importance matrix

Every other public imatrix for this checkpoint is text-only, and not by choice: upstream llama-imatrix has no image path for any model. Collection here runs inside Ember, which routes image tokens through the model's second router bias (exp_probs_b_vl) and implements in-span bidirectional attention visibility. The matrix therefore saw the routing images actually take.

The difference that makes is measurable. Same artifact, same engine, same expert top-k, only the prompts differ:

For scale, this collector against a different engine on the same corpus agrees at 0.99. Every layer's gate/up importance profile moves when the model looks at a photograph. (The two corpora differ in their text as well as in having images, so this conflates modality with prompt style.)

The shipped matrix is a merge, 75 % text / 25 % image. An image-only matrix mis-weights text, and image routing is narrower. Both components are published so you can re-merge at a different weight; 75/25 is a traffic-mix judgement, not a measurement.

• image half: 1,700 requests over 1,684 photographs (COCO val2017 + TextVQA), 387,627 prompt tokens, 73,143 chunks, 1,700/1,700 confirmed to have consumed their image
• text half: 197 prompts, 11,653 chunks
• merged: complete 129/129 expert-tensor coverage, zero uncalibrated experts
• collected at --ds4-expert-top-k 4, the serving configuration
• validated against an independent collector on the same corpus: median cosine 0.9915 across 129 entries

imatrix/vision-corpus-*.json[l] pins the exact 1,700 records, so the image half is reproducible.

Quantization recipe

Identical to the 0731 release: routed experts q2_0_rocmfpx (affine type 107, value = code * scale - offset), token_embd q6_K, attn_output_b q8_0, fused attn_kv dual-scale ROCmFP4, remaining dense/attention ROCmFP4-fast. 2.58 BPW effective.

[!NOTE] Type 107 has no discriminator: the same id is used upstream by a two-scale encoder whose dequant is a different function. This artifact is affine, and the writer records rocmfp2_layout: affine-v1 alongside it.

The abliteration band

Same recipe and the same published direction as the 0731 release: blocks 10–42 plus the 3 DSpark stages, rank-1 direction SHA-256 6e4d8a8f… (MIT, credit drowzeys/keys), lambda 3.5, row-norm preserved, blocks 0–9 untouched.

Per-tensor relative-Frobenius edit: min 0.0495, median 0.0575, max 0.0670 — against 0731's 0.0494 / 0.0563 / 0.0668 on the same 36 tensors.

Probed in the shipped weights: mean ratio 2.161, 33 of 43 blocks above 1.8, blocks 0–9 clean. The production 0731 artifact reads 2.163 on the same probe.

Behavioural validation

Two-armed, policy pre-registered, run at the serving --ds4-expert-top-k 4. Arm B sends each question without the image; any item it answers correctly is cut rather than scored, so a model answering from text priors cannot pass.

No class's arm B is significantly above chance. Spatial cut 8 of 25 items — binary spatial questions are guessable without the image — so its 1.000 stands over the 17 that required seeing it.

Structural validation

• contract: 1374 tensors, 42 classes, all types match
• 129/129 routed-expert tensors at type 107; file_type 106; 175 type expectations
• differential validator: autoregressive, snapshot-restore, DSpark and disk round-trip all token-exact
• compared against the un-abliterated build of the same checkpoint, only two groups of tensors differ: the 43 abliterated attn_output_b, at median 0.057, which matches the edit magnitude measured at abliteration time, and the 129 expert tensors at median 0.203, which is the imatrix change. Every other tensor class is bit-identical.

The DSpark drafter

The 0731 drafter, by measurement. A matched drafter built from this checkpoint's own mtp.* tensors exists and is worse: 0.868 structured-workload acceptance against 0.975, same target, same harness, same window. Swapping the target reproduces the deficit, so it is the MTP head rather than the pairing. Why it is weaker is not established.

Quantization quality — not yet characterized

No perplexity, no KL-divergence against BF16, no capability benchmark, and no throughput measurement for this artifact. The gates above measure whether the vision path works, not what 2.58 BPW costs.

The routed experts arrive from the checkpoint already at 4 bits — DeepSeek ships them as per-32 ue8m0-scaled e2m1, which is MXFP4 — so this is a step down from MXFP4, not from BF16.

Runtime

Built for and tested with Ember. A runtime used instead must implement the deepseek4 graph, ROCmFPx types 100, 101 and affine 107, and image-token routing through exp_probs_b_vl with in-span attention visibility — without the last, images route as text and nothing fails loudly.

--ds4-expert-top-k 4
--vision-mmproj /path/to/mmproj-DeepSeek-V4-Flash-Vision-Exp-F16.gguf
DFLASH_DS4_SPEC=1
DFLASH_DS4_DRAFT=/path/to/DeepSeek-V4-Flash-0731-Abliterated-DSpark-draft-4.25bpw.gguf
DFLASH_DS4_SPEC_Q=6
DFLASH_DS4_FUSED_VERIFY=1

--ds4-expert-top-k 4 is what every measurement here was taken at; the model's own default is 6, and the quality cost of 4 is not measured.

Provenance

deepseek-ai/DeepSeek-V4-Flash-Vision-Exp
  -> rank-1 refusal projection, blocks 10-42 + 3 MTP, lambda 3.5
     direction 6e4d8a8f...37fe0 (MIT, drowzeys/keys)
  -> BF16 GGUF (modular DeepSeek-V4 converter, vision port)
  -> imatrix collected in Ember: 1700 images, then 197 text chunks
     merged 75% text / 25% image
  -> affine expert + STRIX_LEAN ROCmFPx quantization
     archived writer ccb51951...97b7, layout affine-v1
  -> per-tensor type contract check against a known-good GGUF
  -> two-armed behavioural gate at the serving expert top-k

License

Base model licensed by DeepSeek. The importance matrices and corpus manifest published here are MIT, so they can be reused for other quantizations.