Qwen3.8-2.4T-A95B-DSpark-GGUF

Qwen3.8-2.4T-A95B DSpark GGUF

GGUF quantizations of RadixArk DSpark draft model for Qwen3.8-2.4T-A95B.

Use with BeeLlama.cpp, a llama.cpp fork with advanced quantization features.

---

Qwen3.8-Max DSpark speculator

A DSpark speculator for RadixArk/Qwen3.8-2.4T-A95B-NVFP4. DSpark extends the DFlash parallel-draft backbone with a Markov logit-bias head and a per-position confidence head. Trained with SpecForge online distillation from hidden states captured off a live SGLang target engine.

Model specifications

• Base model: RadixArk/Qwen3.8-2.4T-A95B-NVFP4 (92-layer hybrid-attention MoE, hidden size 8192)
• Format: single-file BF16 safetensors, 3,298,141,697 parameters
• Draft: 5 full-attention Qwen3-style GQA layers, hidden size 8192, 64 query heads / 8 KV heads at head_dim 128, block_size=7
• Verification width: 1 current token + 7 draft tokens
• Auxiliary target layers: [3, 19, 35, 55, 71, 87], plus the final post-norm hidden state
• Positional scheme: YaRN factor 32 with original length 8,192 — 262,144 addressable positions, the target's native context limit
• Target weights: embedding and unembedding weights are not included

Acceptance length

Accept length is the mean number of accepted tokens per verify step including the bonus token (SGLang spec_accept_length), averaged over requests. Setting: SGLang DSPARK serving, TP8 on one 8xB300 node, NVFP4 target, BF16 draft (--speculative-draft-model-quantization unquant), block size 7; sampling temperature 0.6, top_k 20, top_p 0.95, thinking enabled; max_new_tokens 2048; 128 prompts per workload (full set where the dataset is smaller). The measurement loop follows z-lab/dflash benchmark.py.

Serving with SGLang

sglang serve \
  --trust-remote-code \
  --model-path RadixArk/Qwen3.8-2.4T-A95B-NVFP4 \
  --tp-size 8 \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path RadixArk/Qwen3.8-2.4T-A95B-DSpark \
  --speculative-dspark-block-size 7 \
  --speculative-draft-attention-backend trtllm_mha