Downloads · 30 days
0
criistii/qwen3-1.7b-int4-rtn32-acc4
qwen3-1.7b-int4-rtn32-acc4 is a machine learning model from criistii. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Qwen3-1.7B exported with the ONNX Runtime GenAI 0.15.2 model builder for the CPU execution provider: every MatMul as MatMulNBits 4-bit, RTN, block size 32, accuracy level 4 (int8 activations). 1.1 GB. Runs with Micros…
Downloads · 30 days
0
Access
Public
Updated Sep 10, 2026
Repo size
1.1 GB
Likes
0
Public
Click a slice to open those files.
.data1.1 GB · 99%
From the Hugging Face model README
Qwen3-1.7B exported with the ONNX Runtime GenAI 0.15.2 model builder for the CPU execution
provider: every MatMul as MatMulNBits 4-bit, RTN, block size 32, accuracy level 4 (int8
activations). 1.1 GB. Runs with Microsoft.ML.OnnxRuntimeGenAI 0.15.x on Android (arm64-v8a) and
iOS, and on desktop CPUs.
Measured on a Samsung Galaxy A26 against the onnx-community cpu-int4-kld-block-128 export of the
same weights (which keeps about half of its layers in 8-bit): loads in 5.6 s instead of 9.2 s,
generates a news summary in 8.4 s instead of 9.4 s, and a Claude judge rated 6 of 20 summaries
usable instead of 3 with fewer invented facts. Details: https://github.com/criistii/AI.POC (docs/FINDINGS.md).
Chat format is Qwen3's ChatML; write an empty <think>\n\n</think>\n\n into the assistant turn to
disable thinking. Greedy decoding recommended for reproducible output.
Built with:
python builder.py -m Qwen/Qwen3-1.7B -o out -p int4 -e cpu --extra_options block_size=32 accuracy_level=4
The tied-embedding GatherBlockQuantized scales were reshaped to [151936, 64] for ONNX Runtime 1.29.