Downloads · 30 days
5.3K
7% of all-time downloads
Qwen3-Reranker-4B-seq-cls-vllm-W4A16
dolfsai/Qwen3-Reranker-4B-seq-cls-vllm-W4A16
Qwen3-Reranker-4B-seq-cls-vllm-W4A16 is a text ranking model from dolfsai. Use it for the text ranking task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
All-time downloads
71.6K
Public
Parameters
4.1B
2.7 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors2.7 GB · 99%
Parameter types
How the weights are stored.
I323.6B · 90%
Base models
Datasets
Model card
From the Hugging Face model README
prudant/Qwen3-Reranker-4B-seq-cls-vllm-fixed-W4A16
Qwen3 4b reranker full vllm adapted 🚀
This is a compressed version of danielchalef/Qwen3-Reranker-4B-seq-cls-vllm-fixed using llm-compressor with the following scheme: W4A16
Serving
python3 -m vllm.entrypoints.openai.api_server --model 'dolfsai/Qwen3-Reranker-4B-seq-cls-vllm-W4A16' --task classify
Important: You MUST read the following guide for correct usage of this model here Guide
Model Details
- Original Model: danielchalef/Qwen3-Reranker-4B-seq-cls-vllm-fixed
- Quantization Method: GPTQ
- Compression Libraries: llm-compressor
- Calibration Dataset: ultrachat_200k (512 samples)
- Optimized For: Inference with vLLM
- License: same as original model
- Task
- Text Ranking
- Type
- qwen3
- License
- apache-2.0
- Languages
- en, es
- Created
- Aug 16, 2025
- Updated
- Aug 18, 2025