llama2-13b-rankllama-teacher

LLaMA2-13B-RankLLaMA-Teacher

Model Description

LLaMA2-13B-RankLLaMA-Teacher is a 13B parameter teacher model designed for neural reranking tasks. This model serves as the foundation for knowledge distillation in the DeAR framework, generating Chain-of-Thought (CoT) reasoning to guide smaller student models.

Model Details

• Model Type: Sequence Classification (Reranking)
• Base Model: LLaMA-2-13B
• Parameters: 13 billion
• Training Data: MS MARCO Passage Ranking
• Purpose: Teacher model for knowledge distillation
• Output: Relevance scores for query-document pairs

Intended Use

This model is intended to:

• Generate training signals for student reranker models
• Provide Chain-of-Thought reasoning for reranking tasks
• Serve as a baseline for evaluating distilled models

Usage

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

# Load model
model_name = "abdoelsayed/llama2-13b-rankllama-teacher"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(
    model_name, 
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model.eval()

# Score a query-document pair
query = "What is machine learning?"
document = "Machine learning is a subset of artificial intelligence that focuses on training algorithms to learn patterns from data."

inputs = tokenizer(
    f"query: {query}",
    f"document: {document}",
    return_tensors="pt",
    truncation=True,
    max_length=512,
    padding=True
)
inputs = {k: v.to(model.device) for k, v in inputs.items()}

with torch.no_grad():
    score = model(**inputs).logits.squeeze().item()
    
print(f"Relevance score: {score}")

Training Details

Training Data

• Dataset: MS MARCO Passage Ranking
• Training Samples: Large-scale query-document pairs with relevance labels

Training Configuration

• Base Model: meta-llama/Llama-2-13b-hf
• Objective: Pointwise ranking with sequence classification
• Hardware: Multi-GPU training (4x A100)
• Precision: Mixed precision (bfloat16)

Hyperparameters

• Learning Rate: 2e-5
• Batch Size: 8 per device
• Epochs: 3
• Max Sequence Length: 512
• Warmup Steps: 1000

Evaluation

This teacher model is evaluated on standard IR benchmarks:

Model Architecture

LLaMA2-13B
    ↓
[Transformer Layers]
    ↓
[Classification Head]
    ↓
Relevance Score

Distillation

This teacher model generates soft labels and CoT reasoning used to train:

• DeAR-8B models (RankNet, CE, Listwise)
• DeAR-3B models (RankNet, CE)

The distillation process uses:

• Temperature: 2.0
• Alpha (KD weight): 0.1
• CoT Dataset: DeAR-COT

Limitations

• Computational Cost: 13B parameters require significant GPU memory (>26GB)
• Inference Speed: Slower than distilled student models
• Domain Specificity: Trained primarily on MS MARCO, may require fine-tuning for other domains

Citation

If you use this model, please cite:

@article{abdallah2025dear,
  title={DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillation},
  author={Abdallah, Abdelrahman and Mozafari, Jamshid and Piryani, Bhawna and Jatowt, Adam},
  journal={arXiv preprint arXiv:2508.16998},
  year={2025}
}

License

MIT License

Related Models

Student Models (Distilled from this teacher):

8B Models:

• DeAR-8B-RankNet
• DeAR-8B-CE
• DeAR-8B-Listwise

3B Models:

• DeAR-3B-RankNet
• DeAR-3B-CE

Dataset:

• DeAR-COT Dataset

More Information

• Repository: GitHub
• Paper: arXiv:2508.16998
• Collection: DeAR Model Collection