Qwen3.8-27B-heretic-dflash2

Qwen3.8-27B-heretic-dflash2 ⚡

Qwen3.8-27B-heretic-dflash2 is a second-generation DFlash 2 speculative drafter tailored and fine-tuned for Qwen 3.8 27B Heretic models (including heretic-org/Qwen3.8-27B-heretic-ara and quantized Apple Silicon MLX variants like MXFP4).

Built upon the state-of-the-art DFlash 2 speculative decoding architecture, this drafter leverages Grouped Dynamic Causal Convolutions (GroupedDynamicCausalConv), Multi-Layer Hidden Representations ($5 \times 5,120 = 25,600$ dims), and Candidate Selector Path Scoring to provide ultra-fast non-autoregressive drafting with exact target model output fidelity.

---

🚀 Live Speculative Benchmarks (Apple Silicon MLX on ml-2)

Evaluated with mlx-vlm serving Qwen3.8-27B-heretic-ara-mxfp4 under standard temperature ($T=0.7$, max_tokens=384):

• Average Speculative Throughput: ~13.0–13.6 tokens/sec on 27B model weights running on unified memory.
• First-Token Acceptance Accuracy: 75.9% on held-out validation benchmark sequences.
• Block Accuracy ($k=7$): 33.7% full 7-token exact block matching.

---

📊 Comparison: Base DFlash2 vs. Heretic DFlash2 (Round 1)

Note: The model is undergoing autonomous multi-round scaling (Rounds 1 → 17) to further specialize token transition probability paths on Heretic / ARA distributions.

---

📈 Multi-Round Continuous Scaling Progress

---

🛠️ Architecture & Innovations in DFlash 2

1. Grouped Dynamic Causal Convolutions (GroupedDynamicCausalConv): Two-stage causal convolutions wrapped around self-attention and feed-forward blocks to capture local phrase structures before and after attention mixing.
2. Candidate Selector Path Scoring: Dual codebooks (predecessor_codebook and successor_codebook) predicting path likelihoods across graph branches.
3. Multi-Layer Target Fusion: Concatenated hidden states from Qwen 3.8 27B layers [5, 19, 33, 47, 61] ($25,600$ feature dimensions).
4. Optimized 8-Token Draft Block: 7 parallel draft tokens per block ($N=8$) minimizing tail verification penalty while maximizing end-to-end token generation rates.

---

💻 Deployment with MLX-VLM on Apple Silicon

Launch mlx_vlm.server with --draft-model and --draft-kind dflash:

python3 -m mlx_vlm.server \
    --host 0.0.0.0 \
    --port 8080 \
    --model /path/to/Qwen3.8-27B-heretic-ara-mxfp4 \
    --trust-remote-code \
    --kv-bits 4 \
    --kv-quant-scheme uniform \
    --kv-group-size 64 \
    --draft-model jfan/Qwen3.8-27B-heretic-dflash2 \
    --draft-kind dflash

Once running, standard OpenAI-compatible /v1/chat/completions API calls will automatically utilize DFlash 2 speculative drafting!