tevatron3-reranker-8b-contrastive

tevatron3-reranker-8b-contrastive

A pointwise passage reranker built on Qwen3-8B (dense), trained with the Tevatron 3.0 toolkit.

• Backbone: Qwen3-8B (dense)
• Training backend: Megatron
• Objective: contrastive
• Parameter efficiency: full fine-tuning
• Training data: RLHN-680K

Scoring contract (important)

This checkpoint is a causal-LM yes/no scorer, saved as Qwen3ForCausalLM (Qwen3MoeForCausalLM for MoE). The relevance score of a (query, passage) pair is the yes/no log-odds at the final prompt token:

score = logit(" yes") − logit(" no")

evaluated right after the prompt suffix "\nIs the passage relevant to the query?". The prompt is "query: {{query}} passage: {{title}} {{text}}\nIs the passage relevant to the query?" and no EOS token is appended.

This differs from the HF sequence-classification rerankers in this release (*-hf, *-lora-hf), which score a single regression logit at EOS.

Usage

Score with Tevatron's reranker eval backend (see the Tevatron repo), or directly:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

name = "brutusxu/tevatron3-reranker-8b-contrastive"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name, dtype=torch.bfloat16).cuda().eval()
yes_id = tok.encode(" yes", add_special_tokens=False)[-1]
no_id  = tok.encode(" no",  add_special_tokens=False)[-1]
prompt = ("query: what is the capital of france "
          "passage: Paris is the capital of France.\n"
          "Is the passage relevant to the query?")
ids = tok(prompt, return_tensors="pt").to("cuda")
with torch.no_grad():
    logits = model(**ids).logits[0, -1]
print((logits[yes_id] - logits[no_id]).item())

Citation

If you use this model, please cite the Tevatron toolkit.