Downloads · 30 days
25
4% of all-time downloads
thehosy/Qwen3.5-397B-A17B-MXFP4-MTP-BF16
Qwen3.5-397B-A17B-MXFP4-MTP-BF16 is a machine learning model from thehosy. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
File-to-file quantization of Qwen/Qwen3.5-397B-A17B with AMD Quark (RTN, no calibration). The MTP (NEXTN draft head) is kept in BF16 so the speculative draft is exact, which maximizes accept length.
Downloads · 30 days
25
4% of all-time downloads
All-time downloads
558
Public
Parameters
222B
236 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors236 GB · 100%
How the weights are stored.
U8206B · 93%
From the Hugging Face model README
File-to-file quantization of Qwen/Qwen3.5-397B-A17B with AMD Quark (RTN, no calibration). The MTP (NEXTN draft head) is kept in BF16 so the speculative draft is exact, which maximizes accept length.
| Component | Precision |
|---|---|
| Routed + shared experts | MXFP4 (W4A4, OCP MX, group size 32) |
| Attention (q/k/v/o proj) | FP8 E4M3 (per-channel weight + dynamic per-token activation) |
GDN / linear-attn, lm_head, MoE gates | BF16 |
| MTP / NEXTN draft head | BF16 |
Size: ~236 GB (≈3.2× smaller than the 752 GB BF16 master).
A companion uniform-FP8-MTP variant is at thehosy/Qwen3.5-397B-A17B-MXFP4-MTP-FP8
(−7 GB, but requires an SGLang patch to load the quantized MTP correctly).
SGLANG_USE_AITER=1 SGLANG_USE_AITER_UNIFIED_ATTN=1 \
python -m sglang.launch_server \
--model-path thehosy/Qwen3.5-397B-A17B-MXFP4-MTP-BF16 \
--trust-remote-code --tp-size 2 \
--attention-backend aiter --linear-attn-backend triton \
--speculative-algorithm EAGLE --speculative-num-steps 3 \
--speculative-eagle-topk 1 --speculative-num-draft-tokens 4 \
--page-size 64 --mem-fraction-static 0.85
No code changes needed for this BF16-MTP variant (the draft loads natively in BF16).
Numbers are hardware/workload specific; accept length is strongly workload-dependent.
Inherits the license of the base model Qwen/Qwen3.5-397B-A17B.