dear-8b-reranker-ranknet-v1

DeAR-8B-Reranker-RankNet-v1

Model Description

DeAR-8B-Reranker-RankNet-v1 is an 8B parameter neural reranker trained with RankNet loss and knowledge distillation. This model is part of the DeAR framework family and achieves strong performance on standard IR benchmarks while being significantly faster than larger teacher models.

Model Details

• Model Type: Pointwise Reranker (Sequence Classification)
• Base Model: LLaMA-3.1-8B
• Parameters: 8 billion
• Training Method: Knowledge Distillation + RankNet Loss
• Teacher Model: LLaMA2-13B-RankLLaMA
• Training Data: MS MARCO
• Precision: BFloat16

Key Features

✅ High Performance: Competitive with 13B teacher on BEIR benchmarks
✅ Fast Inference: 2.2s average latency on standard GPU
✅ Memory Efficient: Fits on single 24GB GPU
✅ Knowledge Distillation: Enhanced with Chain-of-Thought reasoning

Performance

Usage

Quick Start

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

# Load model
model_path = "abdoelsayed/dear-8b-reranker-ranknet-v1"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForSequenceClassification.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16
)
model.eval().cuda()

# Score a query-document pair
query = "What is machine learning?"
document = "Machine learning is a subset of artificial intelligence..."

inputs = tokenizer(
    f"query: {query}",
    f"document: {document}",
    return_tensors="pt",
    truncation=True,
    max_length=228,  # q_max_len(32) + p_max_len(196)
    padding="max_length"
)
inputs = {k: v.cuda() for k, v in inputs.items()}

with torch.no_grad():
    score = model(**inputs).logits.squeeze().item()
    
print(f"Relevance score: {score}")

Batch Reranking

def rerank_documents(query, documents, model, tokenizer, batch_size=64):
    """
    Rerank a list of documents for a query.
    
    Args:
        query: Search query string
        documents: List of (title, text) tuples
        model: Loaded reranker model
        tokenizer: Loaded tokenizer
        batch_size: Batch size for inference
    
    Returns:
        List of (index, score) tuples sorted by relevance
    """
    scores = []
    
    for i in range(0, len(documents), batch_size):
        batch_docs = documents[i:i + batch_size]
        
        # Prepare inputs
        queries = [f"query: {query}"] * len(batch_docs)
        docs = [f"document: {title} {text}" for title, text in batch_docs]
        
        inputs = tokenizer(
            queries,
            docs,
            return_tensors="pt",
            truncation=True,
            max_length=228,
            padding=True
        )
        inputs = {k: v.to(model.device) for k, v in inputs.items()}
        
        # Get scores
        with torch.no_grad():
            logits = model(**inputs).logits.squeeze(-1)
            scores.extend(logits.cpu().tolist())
    
    # Sort by score (descending)
    ranked = sorted(enumerate(scores), key=lambda x: x[1], reverse=True)
    return ranked


# Example usage
query = "When was the Eiffel Tower built?"
documents = [
    ("Eiffel Tower", "The Eiffel Tower was built in 1889 for the World's Fair."),
    ("Paris", "Paris is the capital of France."),
    ("Architecture", "Modern architecture has evolved significantly."),
]

ranking = rerank_documents(query, documents, model, tokenizer)
print(ranking)
# Output: [(0, 8.23), (1, 2.45), (2, -1.87)]

Training Details

Training Data

• Primary Dataset: MS MARCO Passage Ranking

Hardware

• GPUs: 4x NVIDIA A100 (40GB)
• Training Time: ~36 hours
• DeepSpeed: ZeRO Stage 2

Loss Function

RankNet Loss with Knowledge Distillation:

L_total = (1 - α) * L_RankNet + α * L_KD

where:
- L_RankNet: Pairwise ranking loss
- L_KD: KL divergence with teacher (temperature=2)
- α: 0.1 (distillation weight)

Evaluation Results

TREC Deep Learning

BEIR Benchmark

Efficiency

Comparison with Other Models

Model Architecture

Input: "query: [Q] [SEP] document: [D]"
    ↓
LLaMA-3.1-8B Encoder
    ↓
[CLS] Token Representation
    ↓
Linear Classification Head
    ↓
Relevance Score (scalar)

Limitations

• Domain Adaptation: Trained primarily on MS MARCO; may require fine-tuning for specialized domains
• Query Length: Optimized for queries up to 32 tokens
• Document Length: Truncated to 196 tokens; longer documents lose information
• Language: English only
• Numerical Reasoning: Limited capability for queries requiring calculations

Bias and Fairness

This model inherits biases present in:

• Base LLaMA-3.1-8B model
• MS MARCO training data
• Teacher model annotations

Users should evaluate fairness for their specific use cases.

Ethical Considerations

• Search Ranking: Can influence information access and visibility
• Training Data: May contain biased or sensitive content
• Misuse Potential: Should not be used for surveillance or discriminatory ranking

Related Models

DeAR Family:

• DeAR-8B-CE - Binary Cross-Entropy variant
• DeAR-8B-Listwise - Listwise reranking
• DeAR-8B-RankNet-LoRA - LoRA adapter

Teacher:

• LLaMA2-13B-RankLLaMA-Teacher

Dataset:

• DeAR-COT

Citation

@article{abdallah2025dear,
  title={DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillation},
  author={Abdallah, Abdelrahman and Mozafari, Jamshid and Piryani, Bhawna and Jatowt, Adam},
  journal={arXiv preprint arXiv:2508.16998},
  year={2025}
}

License

MIT License

Contact

• GitHub: DataScienceUIBK/DeAR-Reranking
• Paper: arXiv:2508.16998
• Collection: DeAR Models