Downloads · 30 days
318
2% of all-time downloads
Qwen3.8-27B-heretic-dflash
jfan/Qwen3.8-27B-heretic-dflash
Qwen3.8-27B-heretic-dflash is a text generation model from jfan. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.
Qwen3.8-27B-heretic-dflash is a high-performance DFlash speculative drafter trained on multi-layer hidden representations from Qwen 3.8 27B Heretic models (including heretic-org/Qwen3.8-27B-heretic-ara and quantized v…
All-time downloads
13.3K
Public
Parameters
1.7B
62.3 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors3.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- mlx
- License
- apache-2.0
- Model type
- qwen3
- Access
- Public
- Created
- Aug 16, 2026
- Updated
- Aug 25, 2026
- SHA
- 2ea91cb7
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3.8-27B-heretic-dflash ⚡
Qwen3.8-27B-heretic-dflash is a high-performance DFlash speculative drafter trained on multi-layer hidden representations from Qwen 3.8 27B Heretic models (including heretic-org/Qwen3.8-27B-heretic-ara and quantized variants like MXFP4).
By predicting future token blocks in parallel using single forward-pass target hidden representations, this drafter accelerates autoregressive generation on Apple Silicon (MLX) and NVIDIA GPUs by 2.6× to 2.8× with exact target model output fidelity.
🚀 Live Speculative Benchmarks (Apple Silicon MLX)
Evaluated against Qwen3.8-27B-heretic-ara-mxfp4 on real-world prompts with standard temperature:
| Task / Domain | Prompt Scenario | Speculative Acceptance ($k$) | Acceptance Rate (%) | Decode Speed | End-to-End Speedup |
|---|---|---|---|---|---|
| Async Python Coding | Concurrent rate-limited web scraper with retry backoff & error handling | 3.01 tokens / round | 82.05% | 21.9 tok/s | 2.58× faster (vs ~8.5 tok/s baseline) |
| Math & Reasoning | Recursive sequence formula derivation & proof ($\sum_{n=1}^{100} rac{1}{a_n}$) | 4.34 tokens / round | 80.50% | 24.0 tok/s | 2.82× faster (vs ~8.5 tok/s baseline) |
- Draft Acceptance Rate: Over 81% of all proposed draft tokens are verified and accepted.
- First-Token Acceptance Rate ($k \ge 1$): >95%, eliminating stalling during token generation.
📈 Multi-Round Continuous Training Progress
The model is iteratively trained on multi-domain instruction sequences (coding feedback, mathematical reasoning, system architecture, and multi-turn chat) using pure bfloat16 extracted hidden states:
| Checkpoint Stage | Cumulative Sequences | Cumulative Tokens | Held-Out Val Loss | Block Accuracy ($k=15$) | Status |
|---|---|---|---|---|---|
| Untrained Base | 0 | 0 | 7.3247 | 7.7% | Initial z-lab weights |
| Round 1 – 2 | 6,000 | 12.4M | 3.4830 | 19.3% | Baseline Convergence |
| Round 3 – 5 | 15,000 | 31.0M | 3.8210 | 14.8% | Cross-Domain Diversity |
| Round 6 – 8 | 24,000 | 49.5M | 3.6890 | 16.2% | Reasoning & Proofs |
| Round 9 – 10 | 30,000 | 61.9M | 3.3995 | 17.8% | Current Checkpoint (Live) |
| Round 11 – 17 | 51,000 | ~105M | Queued | Queued | Full 50k Step Schedule |
🛠️ Architecture & Specifications
- Drafter Architecture: DFlash Dual-Hidden Non-Autoregressive Drafter
- Target Model: Qwen 3.8 27B Heretic / ARA
- Hidden State Feature Dimension: $5 imes 5,120 = 25,600$ (concatenated target layers in pure
bfloat16) - Context Length: 2,048 tokens + 16 block tokens
- Block Size ($N$): 16 future tokens per speculative step
- Storage Footprint:
3.46 GBsafetensors
💻 How to Use with MLX / MLX-VLM
Launch mlx_vlm.server with --draft-model and --draft-kind dflash:
python3 -m mlx_vlm.server --host 0.0.0.0 --port 8080 --model /path/to/Qwen3.8-27B-heretic-ara-mxfp4 --trust-remote-code --kv-bits 4 --kv-quant-scheme uniform --kv-group-size 64 --draft-model jfan/Qwen3.8-27B-heretic-dflash --draft-kind dflash
Once running, send standard OpenAI-compatible /v1/chat/completions requests to enjoy accelerated 22–24+ tokens/second inference.
- Task
- Text Generation
- Library
- mlx
- Type
- qwen3
- License
- apache-2.0
- Languages
- en, zh, code
- Created
- Aug 16, 2026
- Updated
- Aug 25, 2026