bge-reranker-base-onnx

BGE Reranker Base — ONNX

ONNX export of BAAI/bge-reranker-base, converted using Hugging Face Optimum for faster CPU inference with ONNX Runtime. Drop-in replacement for the original PyTorch model with no code changes beyond the model loader.

Why use this?

The original PyTorch model typically takes 4–5 seconds per batch on CPU. This ONNX export runs ~2x faster with identical ranking quality — verified across multiple query/passage sets with zero ranking order degradation.

Usage

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer
import torch

model = ORTModelForSequenceClassification.from_pretrained("sampoorna42/bge-reranker-base-onnx")
tokenizer = AutoTokenizer.from_pretrained("sampoorna42/bge-reranker-base-onnx")

query = "What causes climate change?"
passage = "Climate change is primarily driven by greenhouse gas emissions from human activities."

inputs = tokenizer(query, passage, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
    score = float(model(**inputs).logits[0][0])

print(f"Relevance score: {score:.4f}")

Download at Build Time (Docker / CI)

from huggingface_hub import snapshot_download
snapshot_download(
    repo_id="your-username/bge-reranker-base-onnx",
    local_dir="./onnx_reranker"
)

No authentication needed — public repo.

Model Details

Performance (CPU benchmark)

Tested on Intel CPU, no GPU, 3 independent query sets with 5–6 passages each:

Scores differ slightly in absolute value (FP32 precision) but relative ranking is always preserved.

License

MIT — same as the original BAAI/bge-reranker-base.