Downloads · 30 days
21
17% of all-time downloads
narcolepticchicken/legalbenchrag-cuad-topone-reranker
legalbenchrag-cuad-topone-reranker is a text ranking model from narcolepticchicken. Use it for the text ranking task on the model card, and read the license before you ship it in a product. It is set up for sentence-transformers. The card lists the license as apache-2.0.
This is a legal retrieval reranker fine-tuned from cross-encoder/ms-marco-MiniLM-L6-v2 on the public orgrctera/legalbenchragcuad retrieval dataset.
Downloads · 30 days
21
17% of all-time downloads
All-time downloads
124
Public
Parameters
22.7M
90.9 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors90.9 MB · 99%
From the Hugging Face model README
This is a legal retrieval reranker fine-tuned from cross-encoder/ms-marco-MiniLM-L6-v2 on the public orgrctera/legalbenchrag_cuad retrieval dataset.
The model scores (question, passage) pairs and is intended to rerank candidate legal/contract passages. It does not generate answers and does not perform first-stage retrieval by itself.
cross-encoder/ms-marco-MiniLM-L6-v2orgrctera/legalbenchrag_cuad0729087702c19847e72c648e8c22f859295a3e0cThe promoted checkpoint is the best local pilot run from a controlled top-one listwise ablation:
10_topone_listwise_pilot_A_temp2p0_2026062310_legalbenchrag_hard_negative_reranker_pilot_20260623T1p0p8_checkpoint_selection1e-5The loss was:
logsumexp(all_candidate_scores / temperature)
- logsumexp(positive_candidate_scores / temperature)
No sigmoid was applied before the listwise loss.
Evaluation used the fixed LegalBenchRAG CUAD pilot split and candidate construction from the local labpack. The final model improved over the previous local incumbent but did not meet the earlier stretch target of MRR@10 > 0.85.
| Model | MRR@10 | Recall@5 | Hits@1 | Top1 / 400 | Top5 / 400 | Over10 |
|---|---|---|---|---|---|---|
| Original frozen baseline | 0.6448 | 0.7700 | n/a | n/a | n/a | n/a |
| Previous local incumbent | 0.7909 | 0.9500 | 0.6875 | 275 | 380 | 8 |
| This checkpoint | 0.8288 | 0.9625 | 0.7350 | 294 | 385 | 1 |
Paired bootstrap versus the previous local incumbent:
+0.0379285714[+0.0188350446, +0.0577749752]1.0Rank distribution for this checkpoint:
{
"1": 294,
"2": 45,
"3": 24,
"4": 13,
"5": 9,
"6": 6,
"7": 1,
"8": 4,
"9": 3,
"25": 1
}
from sentence_transformers import CrossEncoder
model = CrossEncoder("narcolepticchicken/legalbenchrag-cuad-topone-reranker")
pairs = [
(
"Consider the contract; What is the governing law?",
"This Agreement shall be governed by and construed in accordance with the laws of ..."
),
(
"Consider the contract; What is the governing law?",
"The parties shall provide written notice within thirty days ..."
),
]
scores = model.predict(pairs)
Higher scores indicate passages that the model ranks as more likely to answer the query.
Use this model as a second-stage reranker for English contract/legal RAG systems, especially CUAD-like clause retrieval. It expects candidate passages from a separate retriever and ranks them for relevance to the query.
This repository includes:
model.safetensors and Sentence Transformers config/tokenizer filesmetrics.jsonrank_distribution.jsontrain_metrics.jsonrun_manifest.jsonThese local artifacts document the promoted run and evaluation evidence.