ltr-bert-sir

<p align="center"> <img src="logo.svg" alt="ELSIE" width="180" /> </p> <h1 align="center">LTR-BERT SIR</h1> <p align="center"><strong>Pool-restricted bi-encoder for MS MARCO document ranking</strong></p> <p align="center"> <a href="https://huggingface.co/s-m-quadri/ltr-bert-sir"><img src="https://img.shields.io/badge/Model-ltr--bert--sir-yellow?logo=huggingface&label=Model" alt="Model" /></a> <a href="https://huggingface.co/spaces/s-m-quadri/sir-elsie"><img src="https://img.shields.io/badge/Space-sir--elsie-yellow?logo=huggingface&label=Space" alt="Space" /></a> <a href="https://www.npmjs.com/package/@s-m-quadri/ltr-bert-sir-client"><img src="https://img.shields.io/npm/v/@s-m-quadri/ltr-bert-sir-client?logo=npm&logoColor=white&label=npm" alt="npm" /></a> <a href="https://pypi.org/project/ltr-bert-sir-client/"><img src="https://img.shields.io/pypi/v/ltr-bert-sir-client?logo=pypi&logoColor=white&label=PyPI" alt="PyPI" /></a> <a href="https://hub.docker.com/r/smquadri/ltr-bert-sir-elsie"><img src="https://img.shields.io/docker/pulls/smquadri/ltr-bert-sir-elsie?logo=docker&logoColor=white&label=Docker" alt="Docker" /></a> <a href="https://github.com/s-m-quadri/sir-elsie"><img src="https://img.shields.io/badge/Frontend-sir--elsie-181717?logo=github&logoColor=white" alt="GitHub" /></a> <a href="https://www.apache.org/licenses/LICENSE-2.0"><img src="https://img.shields.io/badge/License-Apache_2.0-blue?logo=apache&logoColor=white" alt="License" /></a> </p> <p align="center"> Fine-tuned LTR-BERT-style bi-encoder and scoring code for pool-restricted hybrid document ranking on MS MARCO. </p> <p align="center"> <strong>Authors:</strong> Syed Minnatullah Quadri, Vrishali A. Chakkarwar<br /> <strong>Affiliation:</strong> Department of Computer Science and Engineering, Government Engineering College, Aurangabad, Maharashtra 431001, India<br /> <strong>Contact:</strong> [email protected], [email protected], [email protected]<br /> <strong>Hub:</strong> <a href="https://huggingface.co/s-m-quadri/ltr-bert-sir">s-m-quadri/ltr-bert-sir</a> </p>

Related resources

1. Bi-encoder weights, metrics, and offline LTRBertSIR inference (this repository) at s-m-quadri/ltr-bert-sir on Hugging Face
2. Interactive ELSIE demo and REST API at s-m-quadri/sir-elsie on Hugging Face Spaces
3. JavaScript and TypeScript client at @s-m-quadri/ltr-bert-sir-client on npm
4. Python client at ltr-bert-sir-client on PyPI
5. TypeScript source at ltr-bert-sir-client-js on GitHub
6. Python source at ltr-bert-sir-client-py on GitHub
7. Self-hosted ELSIE API at smquadri/ltr-bert-sir-elsie on Docker Hub
8. Next.js frontend at sir-elsie on GitHub

Try the demo

You do not need a local install to run the full hybrid pipeline on sample corpora.

1. Open sir-elsie.
2. Wait for boot indexing of Alice in Wonderland (default seed) or choose another title from the library catalog.
3. Enter a query, select Hybrid (blend) or Precise (ce_cascade), and inspect ranked passages, timing, and optional evaluation when qrels are loaded.

The Space downloads this checkpoint on first use (~1.3 GB). CPU hardware works; GPU tiers reduce encode and cross-encoder latency.

HTTP clients

Programmatic access to the same API as the Space UI:

Default API origin: https://huggingface.co/spaces/s-m-quadri/sir-elsie. Use LTRBertSIR in this repository for offline bi-encoder scoring without the Space.

Limitations

• Dense scores are pool-restricted: the bi-encoder ranks within a BM25 (or fused) candidate pool, not the full corpus.
• This repository ships inference for the LTR-BERT stage; full MS MARCO reproduction requires external BM25 indices, qrels, and the scripts referenced in the associated publications.
• Cross-encoder reranking in ELSIE uses cross-encoder/ms-marco-MiniLM-L-6-v2 on a short shortlist; it is not bundled as weights in this model repo.

Quick start (notebook)

pip install torch transformers huggingface_hub

import sys
from huggingface_hub import snapshot_download

root = snapshot_download("s-m-quadri/ltr-bert-sir")
sys.path.insert(0, root)

from inference import LTRBertSIR

ranker = LTRBertSIR.from_pretrained(root)

query = "bm25 lexical ranking information retrieval"
docs = [
    "BM25 is a bag-of-words ranking function using term frequency and inverse document frequency.",
    "The weather tomorrow will be sunny with light winds.",
]

print(ranker.score(query, docs[0]))
# 0.8685

print(ranker.rank(query, docs))
# [
#   ("BM25 is a bag-of-words ranking function ...", 0.8685),
#   ("The weather tomorrow will be sunny ...", 0.7953),
# ]

Scores are cosine-style (about [-1, 1]). Rank by relative order inside a candidate list. Paper pipeline uses BM25 first. First run downloads ~1.3 GB (cached). CPU works.

Associated publications

Publication details and bibliographies are listed here on the Hub model card only.

Syed Minnatullah Quadri and Vrishali A. Chakkarwar.
Empirical Analysis of Score Fusion Strategies under Pool-Restricted Dense Encoding for Ad Hoc Document Retrieval.
MSW Management Journal, article 4038.

Syed Minnatullah Quadri and Vrishali A. Chakkarwar.
Lexical First Stage, Pool-Restricted Dense Scores, and Cross-Encoder Reranking on MS MARCO Document Development.
Extended manuscript (in preparation): RRF3 prior, MiniLM CE cascade, out-of-fold fusion; source of the full metrics table below.

System overview

MS MARCO document development (5,193 topics, official qrels):

1. First stage: PySerini BM25@100 (k1=4.46, b=0.82)
2. Bi-encoder: this checkpoint; dense scores only inside each BM25@100 pool
3. Fusion: min-max linear blend with alpha = 0.85 (BM25 weight); RRF / stratified / OOF in the extended manuscript
4. Late stage: MiniLM (cross-encoder/ms-marco-MiniLM-L-6-v2) on top-32 of the RRF3 prior

This Hub repo ships the fine-tuned bi-encoder and scoring code.

Model

Architecture follows Wang et al. (LTR-BERT), with pool-restricted encoding as in the associated papers.

Results (MS MARCO document development)

Macro-averages over 5,193 topics on the BM25@100 pool. Extended manuscript table:

CE vs BM25 on nDCG@10: mean paired difference 0.09541; 95% bootstrap CI [0.08625, 0.10470] (5,193 topics, 4,000 resamples).

Corpus: MS MARCO document ranking v1 (3,213,835 documents). Training: passage triples.train.small. Microsoft MS MARCO. Corpus files are not in this repo.

Intended use

• Bi-encoder stage for the associated publications
• Pool-restricted hybrid reranking: BM25 (or other first stage) then this model, optional fusion / CE
• Query-passage scoring via LTRBertSIR

Citation

@article{quadri2025poolrestricted,
  title   = {Empirical Analysis of Score Fusion Strategies under Pool-Restricted Dense Encoding for Ad Hoc Document Retrieval},
  author  = {Quadri, Syed Minnatullah and Chakkarwar, Vrishali A.},
  journal = {MSW Management Journal},
  url     = {https://mswmanagementj.com/index.php/home/article/view/4038}
}

@article{wang2024ltrbert,
  author  = {Wang, Junmei and Huang, Jimmy X. and Sheng, Jinhua},
  title   = {An Efficient Long-Text Semantic Retrieval Approach via Utilizing Presentation Learning on Short-Text},
  journal = {Complex \& Intelligent Systems},
  volume  = {10},
  pages   = {963--979},
  year    = {2024},
  doi     = {10.1007/s40747-023-01192-3}
}

Cite Bajaj et al. (MS MARCO) when using the dataset.

License

Apache-2.0 for weights and code here. MS MARCO remains under Microsoft research terms.