Downloads · 30 days
9
30% of all-time downloads
MingZwhy/qwen3-8b-hif4-l16-17-18
qwen3-8b-hif4-l16-17-18 is a machine learning model from MingZwhy. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for faquant. The card lists the license as apache-2.0.
This repository contains the pinned FAQUANT Qwen3-8B HiF4 W4A4 checkpoint. It uses QAD followed by BF16 protection of decoder layers 16, 17, and 18.
Downloads · 30 days
9
30% of all-time downloads
All-time downloads
30
Public
Parameters
8.2B
19.6 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors19.6 GB · 100%
How the weights are stored.
BF168.2B · 76%
From the Hugging Face model README
This repository contains the pinned FAQUANT Qwen3-8B HiF4 W4A4 checkpoint. It uses QAD followed by BF16 protection of decoder layers 16, 17, and 18.
本仓库包含固定版本的 FAQUANT Qwen3-8B HiF4 W4A4 权重。模型经过 QAD 训练,并在训练后将第 16、17、18 层恢复为 BF16。
| Benchmark | BF16 | This checkpoint | Loss |
|---|---|---|---|
| MMLU 0-shot, 14,042 questions | 72.9597% | 71.9627% | 0.9970 pp |
| LongBench v1, 21-task macro | 49.4352 | 48.3529 | 1.0824 pp |
LongBench quantizes the complete prompt prefill and first generated token. Later tokens use BF16 decode while reusing the quantized-prefill KV cache. No answer-aware rescoring was used.
Qwen/Qwen3-8BThis is a FAQUANT checkpoint, not a vanilla Transformers checkpoint. The
additional HiF4 metadata must be reconstructed before loading; using
AutoModelForCausalLM.from_pretrained() directly does not reproduce the
quantized model.
Install the public implementation:
git clone https://github.com/MingZwhy/FAQUANT-Public.git
cd FAQUANT-Public
python -m pip install -e .
Download this repository:
from huggingface_hub import snapshot_download
checkpoint = snapshot_download("MingZwhy/qwen3-8b-hif4-l16-17-18")
print(checkpoint)
For MMLU and LongBench commands, see:
Use this model directory as --checkpoint, the original Qwen3-8B directory as
--model, and smooth_qk.pt as --qk-smooth-scales.
model-00001-of-00005.safetensors … model-00005-of-00005.safetensors:
sharded checkpointmodel.safetensors.index.json: tensor-to-shard indexqad_init.json: sanitized FAQUANT loading metadatasmooth_qk.pt: sanitized frozen Smooth-QK calibration artifactmodel-00001-of-00005.safetensors 9306646e20c2fd75ae7dcf9e307c44470f6739d2b4d68771b5634564a3f3c6d6
model-00002-of-00005.safetensors 8b8fb15304deba89cbc1949550b8a2f23964da90801bcaf403c1d27a75861e33
model-00003-of-00005.safetensors a8caf38e74d4a581dab3dbe12059886652333fd0cc37b4a37c440b31f41efad2
model-00004-of-00005.safetensors b323524241f06f021a421bb29fd1306aa8e2a4fa9eef84c3e343d7d739d199e5
model-00005-of-00005.safetensors fd6a97f74a2c07ee68fcd5f4df025a53de3f78bde629dcba0264b5faf70bde3d
smooth_qk.pt 30268adcb9c73a0a496087960f5c658f10da2d80ca4ba1e5b39d9348a338e110
The hashes above identify the evaluated artifact. Retraining with seed 3407 is not claimed to produce byte-identical weights.
Code and aggregate evidence: https://github.com/MingZwhy/FAQUANT-Public/tree/57daafd121b48364b2c5921b6b4a4926babc7208
The FAQUANT additions are Apache-2.0. Use of the checkpoint is also subject to the Qwen3 base model license and terms.