ruri-v3-reranker-310m-lite

ruri-v3-reranker-310m-lite: Zero-Torch & Ultra-Fast Japanese Cross-Encoder Reranker

cl-nagoya/ruri-v3-reranker-310m is an ultra-high-precision Japanese Cross-Encoder reranker based on the ModernBERT-Ja architecture (315M parameters).

ruri-v3-reranker-310m-lite is an optimized deployment package engineered for Zero-Torch runtime (no PyTorch, no Transformers) and zero C++ compilation (pre-built wheels provided).

---

Key Features

1. Zero-Torch Runtime:
   • Zero dependency on heavy deep learning frameworks (torch, torchvision, transformers).
   • Pure runtime dependencies: sentencepiece_lite, onnxruntime / onnxruntime-gpu, numpy, huggingface_hub.
   • Shrinks the entire Python environment footprint from >2.5 GB to ~70 MB.
2. Zero-Compilation Deployment:
   • Pre-built binary wheels for sentencepiece_lite and zero-copy FlatBuffers dictionary (.spm.fb) are bundled.
3. Rigorous Mathematical Equivalence:
   • Max absolute logits difference vs official PyTorch: $< 6.32 \times 10^{-6}$.
   • Top-$k$ rank ordering consistency: 100.0% match.
4. Fast Preprocessing:
   • C++ zero-copy tokenization via SentencePiece Lite is 3.8x – 4.5x faster than Hugging Face AutoTokenizer.
5. Adaptive Hardware Precision:
   • Automatically loads FP16 ONNX on modern GPUs (Turing/Ampere+ with Compute Capability $\ge 7.0$) and FP32 ONNX on Pascal GPUs or CPU.

---

Multi-Tier Size & Precision Options

Transparent on-the-fly streaming Gzip decompression is natively supported in ruri_v3_reranker_lite.py. By default on CPU, precision="auto" loads the ultra-compact int8_full model via 202 MB streaming download.

---

Installation

Install via pip (or uv pip):

# Install the pre-built SentencePiece Lite wheel and runtime dependencies
pip install https://huggingface.co/Chottokun/ruri-v3-reranker-310m-lite/resolve/main/wheels/sentencepiece_lite-0.1.0-cp311-cp311-linux_x86_64.whl onnxruntime numpy huggingface_hub

For GPU acceleration, replace onnxruntime with onnxruntime-gpu.

---

Quickstart

from ruri_v3_reranker_lite import RuriV3RerankerLite

# Initialize model with your desired precision tier:
# - "int8_full" (Recommended for CPU): 301 MB ultra-compact, 1.33x faster, 100% rank preservation
# - "int8": 526 MB dynamic quantized
# - "fp16": 601 MB for modern GPUs (Turing/Ampere+)
# - "fp32": 1,202 MB standard baseline
reranker = RuriV3RerankerLite(
    repo_id="Chottokun/ruri-v3-reranker-310m-lite",
    precision="int8_full",  # Loads the 301 MB ultra-compact model
    device="cpu"
)

query = "日本の首都はどこですか？"
documents = [
    "日本の首都は東京都です。政治・経済の中枢が集約されています。",
    "東京は日本の政治と文化の中心都市であり、多くの観光客が訪れます。",
    "大阪は関西地方の主要都市で、独自の食文化やお笑いで知られています。",
    "明日の天気は全国的に晴れのち曇りとなる見込みです。",
    "ピタゴラスの定理は直角三角形の斜辺の長さを計算するための幾何学の基本法則です。"
]

# 1. Reranking API: Returns sorted results in descending score order
results = reranker.rerank(query, documents, top_k=3, normalize=True)

for rank, item in enumerate(results, start=1):
    print(f"Rank {rank}: Score={item['score']:.4f} (Index {item['index']}) -> {item['document']}")

# Output:
# Rank 1: Score=1.0000 (Index 0) -> 日本の首都は東京都です。政治・経済の中枢が集約されています。
# Rank 2: Score=0.5633 (Index 1) -> 東京は日本の政治と文化の中心都市であり、多くの観光客が訪れます。
# Rank 3: Score=0.0093 (Index 2) -> 大阪は関西地方の主要都市で、独自の食文化やお笑いで知られています。

# 2. Raw Scoring API (Pairs)
pairs = [(query, doc) for doc in documents]
scores = reranker.score(pairs, normalize=True)
print("Relevance Probabilities:", scores)

---

Benchmark & Performance

Tested on Linux x86_64 (CPU, 4 threads, document length ~150 chars):

Two-Stage RAG Architecture Recommendation: Since Cross-Encoder inference runs in $O(N)$ time per query (~1.2s for Top-10 on CPU), we strongly recommend a two-stage hybrid retrieval pipeline:

1. Stage 1 (Coarse Candidate Retrieval): Retrieve Top-20~30 candidate documents using a lightweight Bi-Encoder (e.g., Chottokun/ruri-v3-30m-lite) in $< 10$ ms.
2. Stage 2 (Fine-grained Reranking): Rerank the candidate set using ruri-v3-reranker-310m-lite to extract the most accurate Top-3~5 passages.

---

⚖️ License, Attribution & Citations

This distribution and derived artifacts are licensed under the Apache License 2.0.

Base Model Credits

• Base Model: cl-nagoya/ruri-v3-reranker-310m
• Organization: Nagoya University Natural Language Processing Laboratory (cl-nagoya)
• Authors: Hayato Tsukagoshi, Ryohei Sasano
• Base Model License: Apache License 2.0

Modifications in this Repository

• Converted official Safetensors weights to ONNX (FP32) and ONNX (FP16) with graph attribute sanitization.
• Converted official SentencePiece model into SentencePiece Lite FlatBuffers format (.spm.fb).
• Provided a standalone, zero-torch inference wrapper (ruri_v3_reranker_lite.py).

Base Model Citation

@misc{Ruri,
  title={Ruri: Japanese General Text Embeddings}, 
  author={Hayato Tsukagoshi and Ryohei Sasano},
  year={2024},
  eprint={2409.07737},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2409.07737}, 
}

SentencePiece Lite Credits

• Project: Google SentencePiece Lite
• License: Apache License 2.0