Downloads · 30 days
56
35% of all-time downloads
Qwen2.5-3B-Instruct-int4-ov
Prog-up/Qwen2.5-3B-Instruct-int4-ov
Qwen2.5-3B-Instruct-int4-ov is a text generation model from Prog-up. Use it when you need the model to write or continue text. It is set up for openvino. The card lists the license as apache-2.0.
Qwen/Qwen2.5-3B-Instruct converted to OpenVINO IR with INT4 weight-only quantization (NNCF, groupsize=128, asymmetric, ratio=1.0). Intended for serving with OpenVINO Model Server (OVMS) on CPU / Intel GPU.
All-time downloads
159
Public
Repo size
1.8 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin1.8 GB · 99%
At a glance
- Task
- Text Generation
- Library
- openvino
- License
- apache-2.0
- Model type
- qwen2
- Access
- Public
- Created
- Jun 26, 2026
- Updated
- Jun 26, 2026
- SHA
- 94486d4d
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen2.5-3B-Instruct-int4-ov
Qwen/Qwen2.5-3B-Instruct converted to OpenVINO IR with INT4 weight-only quantization (NNCF, group_size=128, asymmetric, ratio=1.0). Intended for serving with OpenVINO Model Server (OVMS) on CPU / Intel GPU.
Serve with OVMS
docker run -d --rm -p 8000:8000 -v $(pwd)/models:/models --device /dev/dri \
--group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) \
openvino/model_server:2026.2.1-gpu \
--rest_port 8000 --model_repository_path /models \
--source_model Prog-up/Qwen2.5-3B-Instruct-int4-ov --target_device GPU --task text_generation
Then call the OpenAI-compatible endpoint at POST /v3/chat/completions.
Benchmark
~26.1 tok/s decode throughput (INT4, Intel iGPU, OVMS 2026.2.1, 256-token greedy decode).
- Task
- Text Generation
- Library
- openvino
- Type
- qwen2
- License
- apache-2.0
- Created
- Jun 26, 2026
- Updated
- Jun 26, 2026