DeepSeek-V4-Flash-Strix-Halo-GGUF

DeepSeek-V4-Flash-0731 — Strix Halo ROCmFPx GGUF

[!WARNING] This is a custom GGUF for Ember, an ROCmFPx-aware DeepSeek-V4 runtime for AMD Strix Halo (gfx1151). It uses custom tensor types that mainline llama.cpp, LM Studio, and Ollama do not implement. Loading it with an incompatible runtime may fail or produce invalid output.

An abliterated, importance-matrix-calibrated quantization of the official deepseek-ai/DeepSeek-V4-Flash-0731 release, plus its matching DSpark drafter.

[!IMPORTANT] This is the text-only model. Ember's default download is now the vision-capable DeepSeek-V4-Flash-Vision-Exp, which also serves text and ships an image-calibrated importance matrix.

This artifact is not deprecated and has not been deleted. Set EMBER_MODEL_REPO and EMBER_MODEL_REVISION to keep serving it. Its DSpark drafter is the one the vision release ships with, because it measured better than the vision checkpoint's own.

The two models' text quality has not been compared, so neither supersedes the other on evidence.

Ember performance benchmark

Measured 2026-08-22 on one AMD Ryzen AI Max+ 395 with Radeon 8060S (gfx1151) and 128 GB unified memory, against the artifacts published here (target SHA-256 a936e0a5…, drafter 1a01c80e…, both re-verified against the local copies before the run). Runtime was Ember release 2026.8.22 in ember-rocm:7.14, with speculative decoding enabled and per-token instrumentation disabled.

[!IMPORTANT] These measure the production serving configuration: sparse prefill plus DSpark speculative decode, one request at a time. They are not exact-prefill reference numbers and they do not measure quality against BF16.

Every scaling prompt reported zero restored prefix tokens, so each pass evaluated the full prompt. First runs at each shape are included in the medians.

Three greedy 256-token samples produced 34.16 tok/s median decode (33.67–34.16) at 0.989 median DSpark acceptance — one of the three samples reached 1.000 — with speculation active on 3/3.

This release carries two gfx1151 kernel changes: a branch-free UE4M3 scale decode and per-translation-unit selection of WGP vs CU mode. Against the previous published figure of 33.60 tok/s (2026-08-10, commit 8872221, same two model files), that is +1.7% median decode.

[!NOTE] Acceptance is workload-dependent. A greedy 256-token continuation is highly predictable and accepts nearly everything; separate mixed free-form sampling at temperature 0.6 measured ~0.52 mean acceptance on the same pair. Both are real and describe different workloads. The kernel changes in this release are numerically inert: the differential validator reports the autoregressive, DSpark, disk and resident-batch paths all token-exact.

The full benchmark bundle contains the harness, raw JSONL, summary and environment JSON, both SVG sources, and the exact model and runtime identifiers. The 2026-08-10 bundle is retained for comparison.

Quantization quality — not yet characterized

No perplexity, no KL-divergence against BF16, no capability benchmark has been run on this artifact. The numbers above are throughput only. The abliteration has been validated behaviourally (see below), and the tensor layout has been validated structurally, but the quality cost of the 2.58 BPW quantization itself is currently unmeasured. Treat it as unknown rather than as acceptable.

Artifacts

Verify any local copy against the SHA-256 values below.

Target + matching drafter is 102,444,355,040 bytes (95.41 GiB) resident.

The abliteration band

The edit is restricted to the band where the refusal direction actually separates — blocks 10 through 42, plus the 3 DSpark stages, leaving the first ten blocks untouched.

Blocks 0–9 are left untouched. Early layers carry general features and show the weakest harmful/harmless separation, so projecting there costs capability without adding bypass. Abliterating all 46 attention output-B tensors instead of 36 is known to damage agentic coherence: public tuning work on this model family found the all-layers variant reaches full refusal bypass but degrades instruction following, with the characteristic symptom being the model emitting its skill catalog in response to a bare greeting.

The refusal direction is the published rank-1 vector, SHA-256 6e4d8a8f3aa9e21795faab2c5b14d29b019acdf2ddbfbd8238430458a5837fe0 (MIT, credit drowzeys/keys), rather than a locally extracted per-layer set.

Per-tensor relative-Frobenius edit magnitude: min 0.0494, median 0.0566, max 0.0668.

Verification of the abliteration

Each block's attn_output_b was probed directly in the shipped weights by measuring the refusal-direction component against an isotropic baseline (||r^T W|| / (||W||_F / sqrt(n_out))). An untouched tensor sits near 1.0.

Behavioural validation

136 cases, all recorded, greedy (temperature 0, top_p 1, seed 42), reasoning pass disabled:

The bypass set is disjoint from the prompts used to derive the refusal direction. The greeting-versus-skill-catalog case — the symptom that motivated this build — passes. The single coherence failure is a verbosity bound: the model answered correctly but wrapped the answer in a code fence, exceeding a word cap.

This is a self-consistent harness, comparable across builds from this project only. It is not a sealed judge suite and it does not measure capability.

Structural validation

The target was checked against a per-tensor type contract extracted from a GGUF known to run on the target runtime: architecture, tensor count, block range, pinned metadata, and the type of every tensor class.

1,328 tensors, 41 tensor classes, all types match

Type histogram — target: Q4_0_ROCMFP4_FAST 574, F32 535, Q2_0_ROCMFPX 129, Q4_0_ROCMFP4 43, Q8_0 43, I32 3, Q6_K 1. Drafter (82 tensors): F32 46, Q8_0 25, Q4_0_ROCMFP4_FAST 9, F16 2.

The DSpark drafter

The drafter is a separate GGUF, not part of the 43-block target: architecture deepseek4-dflash-draft, 3 blocks, each a full 256-expert MoE block. It is attached at runtime and the two files are never merged.

DFLASH_DS4_SPEC=1
DFLASH_DS4_DRAFT=/path/to/...-Abliterated-DSpark-draft-4.25bpw.gguf
DFLASH_DS4_SPEC_Q=4
DFLASH_DS4_FUSED_VERIFY=1

[!NOTE] Pairing the abliterated drafter is optional. It was measured against the stock drafter on the same target, 16 prompts (8 benign / 8 refusal-triggering), greedy:

drafter	spec runs	mean acceptance	benign	harmful
abliterated (matched)	16/16	0.520	0.605	0.435
stock (mismatched)	16/16	0.508	0.578	0.438

The two are indistinguishable, including on refusal-triggering prompts where a mismatch was predicted to hurt most. No throughput benefit is claimed for the matched drafter. It is published because a drafter that models the same distribution as its target is the defensible default, not because it measured faster. A stock DSpark drafter works.

Speculative decoding is disabled above DFLASH_DS4_SPEC_MAX_CTX.

Note also that the drafter is not interchangeable with a ds4 --mtp DSpark support GGUF: that artifact keeps mtp.* tensor names, while this runtime resolves only blk.*, dflash.* and output_*. They share no tensor names.

Quantization recipe

The base preset is Q4_0_ROCMFP4_STRIX_LEAN, adapted for DeepSeek-V4's fused MLA tensor names:

• all 129 routed-expert gate/up/down tensors: affine Q2_0_ROCMFP2 (custom type 107, 2.5 BPW)
• all 43 fused attn_kv tensors: dual-scale Q4_0_ROCMFP4 (custom type 100)
• 574 remaining dense, attention, shared-expert, indexer, and output tensors: Q4_0_ROCMFP4_FAST (custom type 101)
• all 43 attn_output_b tensors: Q8_0
• token embedding: Q6_K
• norms, biases, scales, and sinks: F32/I32 as appropriate

The affine expert format decodes each block as:

value = code * scale - offset, where code is one of {0, 1, 2, 3}

Its packed blocks remain 2.5 BPW while using both stored scale bytes as an affine scale and offset rather than two symmetric half-block scales.

0731-specific importance matrix

The routed-MoE matrix was regenerated against the official 0731 weights by collecting expert activations on ROCm:

• calibration artifact: DeepSeek-V4-Flash-0731-chat-v2-routed-moe-ds4-rocm.dat
• 4,692 rendered prompts
• 2,897,869 calibration tokens
• 747,650,202 routed-expert observations
• 202,186 quantizer chunks
• complete 129/129 gate/up/down expert-tensor coverage
• no default, zero, duplicate, non-finite, missing, or trailing entries
• imatrix SHA-256: 9095857d3f47f94ad32326ff167e92c50b9a23c6176a1ef68d5af0253c67b3f5

The GGUF embeds the matrix filename, dataset identifier, chunk count, and entry count in its quantization metadata. The downloadable .dat uses the legacy llama.cpp importance-matrix binary format accepted by the ROCmFPx quantizer used for this build.

Runtime validation

Checks this recipe passes end to end:

• header and all 1,328 tensor descriptors parsed successfully
• every special tensor matched the intended type; no missing or unexpected overrides
• final tensor end offset exactly equals the physical file size
• the custom Ember ROCm runtime loaded all tensors into an 86,886.9 MiB GPU buffer on gfx1151
• health, ordinary chat completion, structured tool-call generation, and a tool-result continuation all passed
• DSpark speculative decoding initialized and ran during the smoke tests

The benchmark figures above were measured on this artifact and identify the exact runtime, sampler, and drafter configuration.

Runtime

Built for and tested with Ember (ember-dflash). Any runtime used instead must implement:

• the DeepSeek-V4 (deepseek4) graph
• custom ROCmFPx types 100, 101, and 107
• the affine type-107 CPU reference path and ROCm kernels
• the official DeepSeek-V4-0731 chat template, including DSML tool calls

The model runs without speculative decoding. To use the separate DSpark drafter, the tested runtime configuration was:

DFLASH_DS4_SPEC=1
DFLASH_DS4_DRAFT=/path/to/DeepSeek-V4-Flash-0731-Abliterated-DSpark-draft-4.25bpw.gguf
DFLASH_DS4_SPEC_Q=4
DFLASH_DS4_FUSED_VERIFY=1

A matching abliterated drafter is now published in this repository (see Artifacts). The upstream original is deepseek-ai/DeepSeek-V4-Flash-DSpark, and the GGUF used during validation came from Lucebox/DeepSeek-V4-Flash-DSpark-Drafter-GGUF.

Provenance

deepseek-ai/DeepSeek-V4-Flash-0731
revision 9e165c30e2704aec5d9d593cce3eebd58bbef1cb
  -> rank-1 refusal projection, blocks 10-42 + 3 MTP, lambda 3.5
     direction 6e4d8a8f...37fe0 (MIT, drowzeys/keys)
  -> BF16 GGUF (modular DeepSeek-V4 converter)
  -> 0731 ROCm activation collection (imatrix)
  -> affine expert + STRIX_LEAN ROCmFPx quantization
  -> per-tensor type contract check against a known-good GGUF

The base model's improved 0731 Jinja chat template is retained in the GGUF.

License

This artifact inherits the DeepSeek model license. See the base model card and license for its terms.