Downloads · 30 days
33
6% of all-time downloads
Qwen3-4B-Autoround-Safetensors
simaai/Qwen3-4B-Autoround-Safetensors
Qwen3-4B-Autoround-Safetensors is a text generation model from simaai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Pre-LLiMa Hugging Face checkpoint based on Qwen/Qwen3-4B at source revision 1cfa9a7208912126459214e8b04321603b3df60c. It remains subject to the base model's license, intended use, and limitations.
All-time downloads
598
Public
Parameters
4.4B
5.7 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors2.8 GB · 99%
Parameter types
How the weights are stored.
I324B · 91%
Base models
Model card
From the Hugging Face model README
Qwen3-4B Autoround
Pre-LLiMa Hugging Face checkpoint based on Qwen/Qwen3-4B at source revision 1cfa9a7208912126459214e8b04321603b3df60c. It remains subject to the base model's license, intended use, and limitations.
Quantization
| Component | Method | Weight format | Details |
|---|---|---|---|
| Decoder Linear layers | AutoRound | symmetric INT4, G256 | 252 targets; lm_head excluded |
lm_head | GPTQ | symmetric INT4, G256 | static act-order, block size 128, dampening 0.01 |
| Mixed-precision exceptions | — | source dtype | non-Linear parameters remain at source precision |
Calibration used HuggingFaceH4/ultrachat_200k (dataset revision not captured), train_sft[:512], rendered with the source chat template. Token IDs were deterministically concatenated and packed into 512 full 1024-token spans (no shuffle); AutoRound used 200 iterations and batch size 1.
Evaluation
Matched full WikiText evaluation used the wikitext task backed by EleutherAI/wikitext_document_level, wikitext-2-raw-v1, with no example limit and the same evaluator settings for source and quantized checkpoints.
| Checkpoint | Word perplexity |
|---|---|
Source (Qwen/Qwen3-4B) | 18.454564 |
| This UltraChat-calibrated checkpoint | 20.196137 |
The source-to-quantized change is +1.741573 PPL (+9.437%). quantize.py completed finite-scale validation and a Transformers chat-generation smoke test.
Reproduction
This directory includes the exact quantize.py, recipe.yaml, and versions.txt.
python quantize.py --model-path /path/to/models--Qwen--Qwen3-4B --output-dir /path/to/output
Environment
Exact Python, CUDA, Torch, Transformers, llmcompressor, AutoRound, and compressed-tensors versions are recorded in versions.txt.
Deployment
This is the pre-LLiMa quantized Hugging Face artifact. Compile it separately for the target Sima.ai platform and keep compiler output separate.
Limitations
Quantization quality can vary by language, domain, prompt format, context length, and deployment runtime; validate the intended workload independently.
- Task
- Text Generation
- Library
- transformers
- Type
- qwen3
- License
- apache-2.0
- Languages
- multilingual
- Created
- Jul 16, 2026
- Updated
- Sep 1, 2026