Downloads · 30 days
193
8% of all-time downloads
Qwen3-Reranker-4B-F32-GGUF
prithivMLmods/Qwen3-Reranker-4B-F32-GGUF
Qwen3-Reranker-4B-F32-GGUF is a text ranking model from prithivMLmods. Use it for the text ranking task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and…
All-time downloads
2.3K
Public
Repo size
58.2 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf58.2 GB · 100%
At a glance
- Task
- Text Ranking
- Library
- transformers
- License
- apache-2.0
- Model type
- qwen3
- Access
- Public
- Created
- Jun 26, 2025
- Updated
- Jun 26, 2025
- SHA
- cab28a3d
Base models
Model card
From the Hugging Face model README
Qwen3-Reranker-4B-F32-GGUF
Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and reasoning skills of its foundational model. The Qwen3 Embedding series represents significant advancements in multiple text embedding and ranking tasks, including text retrieval, code retrieval, text classification, text clustering, and bitext mining.
Model Files
| Filename | Size | Format | Description |
|---|---|---|---|
| Qwen3-Reranker-4B.BF16.gguf | 8.05 GB | BF16 | Brain Float 16-bit quantization |
| Qwen3-Reranker-4B.F16.gguf | 8.05 GB | F16 | Half precision (16-bit) floating point |
| Qwen3-Reranker-4B.F32.gguf | 16.1 GB | F32 | Full precision (32-bit) floating point |
| Qwen3-Reranker-4B.Q2_K.gguf | 1.67 GB | Q2_K | 2-bit quantization with K-quant |
| Qwen3-Reranker-4B.Q3_K_L.gguf | 2.24 GB | Q3_K_L | 3-bit quantization (Large) with K-quant |
| Qwen3-Reranker-4B.Q3_K_M.gguf | 2.08 GB | Q3_K_M | 3-bit quantization (Medium) with K-quant |
| Qwen3-Reranker-4B.Q3_K_S.gguf | 1.89 GB | Q3_K_S | 3-bit quantization (Small) with K-quant |
| Qwen3-Reranker-4B.Q4_K_M.gguf | 2.5 GB | Q4_K_M | 4-bit quantization (Medium) with K-quant |
| Qwen3-Reranker-4B.Q4_K_S.gguf | 2.38 GB | Q4_K_S | 4-bit quantization (Small) with K-quant |
| Qwen3-Reranker-4B.Q5_K_M.gguf | 2.89 GB | Q5_K_M | 5-bit quantization (Medium) with K-quant |
| Qwen3-Reranker-4B.Q5_K_S.gguf | 2.82 GB | Q5_K_S | 5-bit quantization (Small) with K-quant |
| Qwen3-Reranker-4B.Q6_K.gguf | 3.31 GB | Q6_K | 6-bit quantization with K-quant |
| Qwen3-Reranker-4B.Q8_0.gguf | 4.28 GB | Q8_0 | 8-bit quantization |
Recommended Usage for Reranking Tasks
- Q4_K_M or Q5_K_M: Optimal balance for most reranking applications
- Q6_K or Q8_0: Higher precision for critical ranking accuracy
- Q3_K_M: Good performance with reduced memory footprint
- F16 or BF16: Maximum reranking precision, requires more VRAM
- F32: Highest precision for research and benchmarking
Quants Usage
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):
- Task
- Text Ranking
- Library
- transformers
- Type
- qwen3
- License
- apache-2.0
- Languages
- en
- Created
- Jun 26, 2025
- Updated
- Jun 26, 2025