nyaya-reranker-mini-v1

Nyaya-Reranker-Mini-v1 — a 118M cross-encoder for Indian statute sections

cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 (Apache-2.0) fine-tuned to score a citizen's question against a candidate section of current Indian law. It reorders the BM25 top-20 from NyayaLabs98/nyaya-statute-db in the Nyaya retriever. One fifth the size of the BAAI/bge-reranker-v2-m3 the project also supports, and weaker than it.

⚖️ Not legal advice. This model ranks statute sections. What to do about them is a question for an advocate enrolled under the Advocates Act, 1961.

Numbers

Full-hit recall (every gold section of a question inside the top-k) after reranking the BM25 top-20, on the Eval-v1 questions never used to tune retrieval (n=118), scripts/15_retrieval_recall.py --rerank:

Report: reports/retrieval_recall_rerank_mini.json. The 3.2 s CPU latency is why the browser demo still runs BM25 only.

Training

• Examples: 21,668 (question, section text, label) triples from 4,412 training questions: every gold section as a positive and four of the twenty BM25 hard negatives as negatives (scripts/41_build_retriever_pairs.py; all Eval-v1 questions excluded).
• Recipe: CrossEncoder.fit, binary relevance, batch 32, 1 epoch, 200 warm-up steps, mixed precision, one Kaggle T4 (272 s). Final training loss 0.30.
• Passage text: nyaya.rerank.passage_text (act name, section title, first 1,600 characters of the section).

Use

from sentence_transformers import CrossEncoder
model = CrossEncoder("NyayaLabs98/nyaya-reranker-mini-v1", max_length=512)
scores = model.predict([("police FIR nahi likh rahi, kya karu?", "Bharatiya Nagarik Suraksha Sanhita, 2023 — Section 173 ...")])

In the repository: python scripts/15_retrieval_recall.py --rerank NyayaLabs98/nyaya-reranker-mini-v1 --rerank-depth 20.

Limits

• Bounded by its candidates: nothing outside the BM25 top-20 can be recovered.
• Below bge-reranker-v2-m3 at k=1 by 6.8 points on n=118; use bge when you can afford it.
• Trained on 27 acts plus the Constitution only.