mist-reranker-150m

mist-reranker-150m

A compact English cross-encoder reranker for the second stage of a RAG / search pipeline. Given a query and a candidate passage, it outputs a relevance score to re-sort the top-k from a first-stage retriever (BM25 or a bi-encoder).

Built on answerdotai/ModernBERT-base (~150M) and fine-tuned on olaverse/reranker-general-en-llm-judged. Competitive with purpose-built 150M ModernBERT rerankers on QA/fact retrieval, at equal or smaller size.

📄 Model details

🏃 How to run

pip install -U sentence-transformers

from sentence_transformers import CrossEncoder

model = CrossEncoder("olaverse/mist-reranker-150m")

query = "What causes ocean tides?"
passages = [
    "Tides are caused by the gravitational pull of the Moon and Sun on Earth's oceans.",
    "The Pacific Ocean is the largest of Earth's oceanic divisions.",
    "Tidal energy is a renewable power source generated from the movement of tides.",
]
scores = model.predict([(query, p) for p in passages])
for s, p in sorted(zip(scores, passages), reverse=True):
    print(f"{s:.3f}  {p}")

📈 Performance

NanoBEIR nDCG@10, all models run through the same evaluation harness (CrossEncoderNanoBEIREvaluator). Every task is zero-shot — held out from training for every model. Tasks are split into in-domain QA/fact and out-of-domain argument retrieval.

On QA/fact retrieval, mist-reranker-150m (0.628) is within ~0.005 of granite (0.633) and ahead of bge-reranker-base (0.566) — 1.8× larger — and MiniLM-L12. It was trained on a fully-disclosed, MS-MARCO-free data mix; the comparison models use larger undisclosed training data. The parity is achieved with cleaner, verifiable data provenance.

🧪 Training

• Base: answerdotai/ModernBERT-base, trained from scratch into a reranker.
• Data: LLM-judged graded 0–3 relevance labels (Qwen2.5-72B, UMBRELA-style rubric), validated at Cohen's κ (quadratic) = 0.491 vs TREC-DL human qrels.
• Loss: BinaryCrossEntropyLoss on labels normalized to [0,1]; 2 epochs, best checkpoint by NanoBEIR mean nDCG@10.
• Raw ModernBERT-base scored 0.108 mean nDCG@10 before training; the dataset does effectively all the reranking learning.

License

Released under Apache-2.0.

Citation

@misc{mist-reranker-150m,
  title  = {mist-reranker-150m},
  author = {Olaverse},
  year   = {2026},
  url    = {https://huggingface.co/olaverse/mist-reranker-150m}
}