Downloads · 30 days
2K
0% of all-time downloads
Qwen3-Embedding-4B-W4A16-G128
boboliu/Qwen3-Embedding-4B-W4A16-G128
Qwen3-Embedding-4B-W4A16-G128 is a feature extraction model from boboliu. Use it when you need embeddings to search or compare text. It is set up for sentence-transformers. The card lists the license as apache-2.0.
GPTQ Quantized Qwen/Qwen3-Embedding-4B with THUIR/T2Ranking and m-a-p/COIG-CQIA for calibration set.
All-time downloads
2.5M
Public
Parameters
4.1B
2.7 GB on disk
Likes
5
Public
Repo makeup
Click a slice to open those files.
.safetensors2.7 GB · 99%
Parameter types
How the weights are stored.
I323.6B · 90%
Base models
Model card
From the Hugging Face model README
Qwen3-Embedding-4B-W4A16-G128
GPTQ Quantized Qwen/Qwen3-Embedding-4B with THUIR/T2Ranking and m-a-p/COIG-CQIA for calibration set.
What's the benefit?
VRAM Usage: 17430M -> 11000M (w/o FA2).
What's the cost?
~0.72% lost in C-MTEB.
Evaluation performed with official code.
| C-MTEB | Param. | Mean(Task) | Mean(Type) | Class. | Clust. | Pair Class. | Rerank. | Retr. | STS |
|---|---|---|---|---|---|---|---|---|---|
| multilingual-e5-large-instruct | 0.6B | 58.08 | 58.24 | 69.80 | 48.23 | 64.52 | 57.45 | 63.65 | 45.81 |
| bge-multilingual-gemma2 | 9B | 67.64 | 68.52 | 75.31 | 59.30 | 86.67 | 68.28 | 73.73 | 55.19 |
| gte-Qwen2-1.5B-instruct | 1.5B | 67.12 | 67.79 | 72.53 | 54.61 | 79.5 | 68.21 | 71.86 | 60.05 |
| gte-Qwen2-7B-instruct | 7.6B | 71.62 | 72.19 | 75.77 | 66.06 | 81.16 | 69.24 | 75.70 | 65.20 |
| ritrieve_zh_v1 | 0.3B | 72.71 | 73.85 | 76.88 | 66.5 | 85.98 | 72.86 | 76.97 | 63.92 |
| Qwen3-Embedding-4B | 4B | 72.27 | 73.51 | 75.46 | 77.89 | 83.34 | 66.05 | 77.03 | 61.26 |
| This Model | 4B-W4A16 | 71.75 | 73.05 | 75.43 | 77.51 | 83.04 | 65.73 | 76.15 | 60.47 |
How to use it?
pip install compressed-tensors optimum and auto-gptq / gptqmodel, then goto the official usage guide.
- Task
- Feature Extraction
- Library
- sentence-transformers
- Type
- qwen3
- License
- apache-2.0
- Created
- Jun 6, 2025
- Updated
- Jun 7, 2025