Downloads · 30 days
26
6% of all-time downloads
Phi-4-mini-instruct-Autoround-Safetensors
simaai/Phi-4-mini-instruct-Autoround-Safetensors
Phi-4-mini-instruct-Autoround-Safetensors is a text generation model from simaai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This is a post-training quantized version of microsoft/Phi-4-mini-instruct, prepared as a Hugging Face checkpoint for LLiMa compilation and deployment on Sima.ai hardware.
All-time downloads
453
Public
Parameters
4.5B
10.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.5 GB · 99%
Parameter types
How the weights are stored.
I323.8B · 86%
Base models
Model card
From the Hugging Face model README
Phi-4-mini-instruct Autoround
This is a post-training quantized version of
microsoft/Phi-4-mini-instruct,
prepared as a Hugging Face checkpoint for LLiMa compilation and deployment on
Sima.ai hardware.
This checkpoint remains subject to the source model's license, intended-use guidance, and limitations.
Quantization
| Component | Method | Weight format | Details |
|---|---|---|---|
| Decoder Linear layers | AutoRound | symmetric INT4, G256 | lm_head excluded; 200 iterations; batch size 1 |
lm_head | GPTQ | symmetric INT8, per-channel | static act-order |
| Mixed-precision exceptions | — | source dtype | non-Linear parameters remain at source precision |
Calibration used HuggingFaceH4/ultrachat_200k, split train_sft[:512].
Conversations were rendered with the source chat template and packed into 512
sequences of 1,024 tokens without shuffling.
Evaluation
Full wikitext-2-raw-v1 evaluation used the wikitext lm-eval task with no
example limit and matched source/quantized settings on 2026-07-31.
| Checkpoint | Word perplexity | Status |
|---|---|---|
microsoft/Phi-4-mini-instruct | 11.713516 | Full run |
| This quantized checkpoint | 12.560528 | Full run |
| Absolute degradation | 0.847012 | Lower is better |
| Relative degradation | 7.231% | 100 * (quantized / source - 1) |
Finite-scale validation and a Transformers chat-generation smoke test passed.
Reproduction
This directory includes the model-specific quantize.py, recipe.yaml, and
versions.txt.
python quantize.py \
--model-path /path/to/models--microsoft--Phi-4-mini-instruct \
--output-dir /path/to/Phi-4-mini-instruct-Autoround-Safetensors
Environment
Exact Python, CUDA, Torch, Transformers, llmcompressor, AutoRound, and
compressed-tensors versions are recorded in versions.txt.
Deployment
This is the pre-LLiMa quantized Hugging Face artifact. Compile it separately for the target Sima.ai platform and keep compiler output separate.
Limitations
Quantization quality can vary by language, domain, prompt format, context length, and deployment runtime. Validate the intended workload independently.
- Task
- Text Generation
- Library
- transformers
- Type
- phi3
- License
- mit
- Languages
- multilingual
- Created
- Jul 15, 2026
- Updated
- Sep 1, 2026