Qwen3-Reranker-4B-seq-cls-vllm-fixed

Qwen3-Reranker-4B-seq-cls-vllm-fixed

This is a fixed version of the Qwen3-Reranker-4B model converted to sequence classification format, optimized for use with vLLM.

Model Description

This model is a pre-converted version of Qwen/Qwen3-Reranker-4B that:

• Has been converted from CausalLM to SequenceClassification architecture
• Includes proper configuration for vLLM compatibility
• Provides ~75,000x reduction in classification head size
• Offers ~150,000x fewer operations per token compared to using the full LM head

Key Improvements

The original converted model (tomaarsen/Qwen3-Reranker-4B-seq-cls) was missing critical vLLM configuration attributes. This version adds:

{
  "classifier_from_token": ["no", "yes"],
  "method": "from_2_way_softmax",
  "use_pad_token": false,
  "is_original_qwen3_reranker": false
}

These configurations are essential for vLLM to properly handle the pre-converted weights.

Usage with vLLM

vllm serve danielchalef/Qwen3-Reranker-4B-seq-cls-vllm-fixed \
    --task score \
    --served-model-name qwen3-reranker-4b \
    --disable-log-requests

Python Example

from vllm import LLM

llm = LLM(
    model="danielchalef/Qwen3-Reranker-4B-seq-cls-vllm-fixed",
    task="score"
)

queries = ["What is the capital of France?"]
documents = ["Paris is the capital of France."]

outputs = llm.score(queries, documents)
scores = [output.outputs.score for output in outputs]
print(scores)

Performance

This model performs identically to the original Qwen3-Reranker-4B when used with proper configuration, while providing significant efficiency improvements:

• Memory: ~600MB → ~8KB for classification head
• Compute: 151,936 logits → 1 logit per forward pass
• Speed: Faster inference due to reduced computation

Technical Details

• Architecture: Qwen3ForSequenceClassification
• Base Model: Qwen/Qwen3-Reranker-4B
• Conversion Method: from_2_way_softmax (yes_logit - no_logit)
• Model Size: 4B parameters
• Task: Reranking/Scoring

Citation

If you use this model, please cite the original Qwen3-Reranker:

@misc{qwen3reranker2024,
  title={Qwen3-Reranker},
  author={Qwen Team},
  year={2024},
  publisher={Hugging Face}
}

License

Apache 2.0 (inherited from the base model)