Downloads · 30 days
19
4% of all-time downloads
Phi-4-mini-instruct-W4A16
useful-quants/Phi-4-mini-instruct-W4A16
Phi-4-mini-instruct-W4A16 is a text generation model from useful-quants. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Ampere-friendly serving build of microsoft/Phi-4-mini-instruct. Text-side linears are compressed-tensors W4A16.
All-time downloads
476
Public
Parameters
4.5B
4.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors4.1 GB · 100%
Parameter types
How the weights are stored.
I323.2B · 72%
Try a prompt
Base models
Model card
From the Hugging Face model README
Phi-4-mini-instruct W4A16
Ampere-friendly serving build of microsoft/Phi-4-mini-instruct.
Text-side linears are compressed-tensors W4A16.
Stock proof
docker run --rm -it \
--gpus all \
--ipc=host \
-p 8001:8000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
vllm/vllm-openai:latest \
microsoft/Phi-4-mini-instruct \
--served-model-name Phi-4-mini-instruct-stock \
--dtype bfloat16 \
--max-model-len 4096 \
--gpu-memory-utilization 0.7
Serve the packaged artifact
docker run --rm -it \
--gpus all \
--ipc=host \
-p 8002:8000 \
-v /path/to/Phi-4-mini-instruct-W4A16:/model \
-v ~/.cache/huggingface:/root/.cache/huggingface \
vllm/vllm-openai:latest \
/model \
--served-model-name Phi-4-mini-instruct-W4A16 \
--dtype bfloat16 \
--quantization compressed-tensors \
--max-model-len 4096 \
--gpu-memory-utilization 0.7
Smoke test
python verify.py --url http://localhost:8002/v1/completions
Notes
- Best fit: RTX 30xx/40xx Ampere cards.
- The package is text-only and stays compatible with clean vLLM.
- Task
- Text Generation
- Library
- transformers
- Type
- phi3
- License
- mit
- Created
- Apr 30, 2026
- Updated
- Apr 30, 2026