DeepSeek-V4-Flash-JANGTQ-K

<p align="center"><img src="jangq-logo.png" width="160"/></p> <p align="center"> <a href="https://vmlx.net"><img src="vmlx-app.png" alt="vMLX — run JANG models on Apple Silicon" width="820"></a> </p> <h3 align="center">⚡ All JANG models are meant to be run in <a href="https://vmlx.net">vMLX</a></h3>

DeepSeek-V4-Flash-JANGTQ-K

JANGTQ-K — the canonical max-quality JANG quantization of deepseek-ai/DeepSeek-V4-Flash for Apple Silicon, in the ~80 GiB tier.

What this is

The first DSV4-Flash JANGTQ bundle that survives the relaxed multi-turn reasoning_effort=max quality-chat gate. It pairs:

• 2-bit MXTQ routed experts on 38 of 43 layers,
• 4-bit MXTQ routed experts on the 5 layers that lost the most quality at 2-bit (L23, L25, L28, L34, L36), selected from a real-activation probe over all 43 layers,
• 8-bit affine for every non-routed module (attention, shared expert, Compressor, Indexer, embed, lm_head),
• float32 preserved for HC controls, attention sinks, and APE,
• MTP head dropped (DSV4-Flash ships one MTP module of ~6.5 B params; not used at inference).

Live Quality

Passed: 4-turn reasoning_effort=max chat gate. Correct arithmetic, memory recall (remembered HARBOR-17 and CERULEAN), final 3-bullet format honored, no loops, no empty visible answers. Paged DSV4 cache hit on turns 2–4 with 399 tokens saved per hit; block-L2 wrote composite DSV4 blocks. Speed: 14.6 – 17.6 tok/s on M3 Ultra Mac Studio.

Not yet passed: strict exact-copy gate. Short exact-marker rows can flip to neighboring BPE tokens. The failure reproduces with prefix-cache bypass and is not a UI, gateway, streaming, or disk-cache artifact. Raw-max (VMLINUX_DSV4_RAW_MAX=1) had a 2-turn smoke only and is not advertised as fully cleared.

Quantization Recipe

JANG vs MLX comparison

Benchmarks pending. To follow Eric's README standard (no TBD), this section will be filled in before any social promotion. The intended axes are:

• MMLU (reasoning + no-reasoning)
• HumanEval pass@1
• Korean / Chinese subset accuracy
• Decode tok/s on M3 Ultra, M5 Max
• Bundle size on disk

The MLX comparison baseline will be the smallest MLX-affine quant of DSV4-Flash that loads on the same hardware; if MLX has no published DSV4 quant at this size class, that absence will be stated.

Architecture (for downstream tooling)

• 43 decoder layers, no MTP.
• MoE: 256 routed + 1 shared per layer, top-6, routed_scaling_factor=1.5, topk_method=noaux_tc.
• MLA: q_lora_rank=1024, o_lora_rank=1024, head_dim=512, qk_rope_head_dim=64.
• Sparse Indexer: index_n_heads=64, index_head_dim=128, index_topk=512, compress_rope_theta=160000.
• HC controls: hc_mult=4, hc_sinkhorn_iters=20, hc_eps=1e-6.
• Sliding window 128; SwiGLU clip 10.0; rope_theta=10000.
• Context length 1,048,576 with YaRN.
• Reasoning modes chat / thinking; reasoning_effort ∈ {max, high, null}.
• Tool calling: DSML parser (<｜DSML｜> block).

Runtime contract

Bundle metadata pins:

• DSV4 native model family deepseek_v4, batch path DSV4BatchGenerator.
• Cache schema deepseek_v4_v7; layers 0,1 = KVCache, 2..42 = DeepseekV4Cache.
• Generic TurboQuant KV: off for DSV4 (composite cache).
• DSV4 pool quant: off by default; opt-in only with DSV4_POOL_QUANT=1.
• Paged cache block size: 256 (loader upgrades stale 64-token settings).
• Block disk L2: on.
• Chunked prefill: off; single-shot prefill is the safe path.

JANGTQ runtime sidecar

jangtq_runtime.safetensors carries the JANGTQ MXTQ codebook + signs tables required by the Swift loader. Six tensors total (codebook.{2048,4096}.{2,4} + signs.{2048,4096}.42).

Sampling defaults

temperature=0.6, top_p=0.95
repetition_penalty=1.0 (1.0 thinking, 1.05 chat)
max_new_tokens=4096

Files

• config.json — DSV4 HF config.
• jang_config.json — JANG profile, recipe, bit plan, runtime requirements, chat encoder, sampling defaults, lineage.
• model-00001-of-00080.safetensors … model-00080-of-00080.safetensors — sharded weights.
• model.safetensors.index.json — tensor → shard map (2610 keys).
• jangtq_runtime.safetensors — JANGTQ MXTQ runtime sidecar.
• tokenizer.json, tokenizer_config.json — preserved upstream tokenizer.
• generation_config.json — HF defaults.
• encoding/encoding_dsv4.py — DSV4 chat encoder (Python).
• LICENSE — MIT, upstream.

Lineage

Converter variant V3 (now collapsing into K), plan routed_only_worst5_23_25_28_34_36 (sha256 3db0b31fe6f1b19d3e00cfdd15572ebf3af950ef25e1e8622e1f2791b1977619). Originally staged as DeepSeek-V4-Flash-JANGTQ-V3-WORST5-F32; renamed to JANGTQ-K on 2026-05-11 as the canonical DSV4 max-quality tier.

Contact

[email protected]