Downloads · 30 days
14
4% of all-time downloads
bge-reranker-v2-m3-int8-onnx
csylabs/bge-reranker-v2-m3-int8-onnx
bge-reranker-v2-m3-int8-onnx is a text ranking model from csylabs. Use it for the text ranking task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Dynamic INT8 quantization of BAAI/bge-reranker-v2-m3 for fast CPU reranking via Hugging Face Text Embeddings Inference (TEI).
All-time downloads
347
Public
Repo size
587 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.onnx570 MB · 97%
At a glance
- Task
- Text Ranking
- License
- mit
- Model type
- xlm-roberta
- Access
- Public
- Created
- Jun 20, 2026
- Updated
- Jun 20, 2026
- SHA
- eaf5072d
Base models
Model card
From the Hugging Face model README
bge-reranker-v2-m3 · INT8 ONNX (AVX-512 VNNI)
Dynamic INT8 quantization of BAAI/bge-reranker-v2-m3 for fast CPU reranking via Hugging Face Text Embeddings Inference (TEI).
- Recipe:
optimumORTQuantizer ·AutoQuantizationConfig.avx512_vnni(is_static=False, per_channel=False)(dynamic, no calibration). - Why: ~2.5× faster reranking on AVX-512-VNNI CPUs, ~4× smaller (543 MB vs 2.3 GB fp32), no measured ranking-quality loss on our legal/finance retrieval gold (pravo/cfa/border: zero-question delta vs fp32).
- Serving: TEI
cpu-1.9.3+—onnx/model.onnxis the int8 graph; config + tokenizer at root.
Quantized by ЛИИ / csylabs for the «Доверенный ИИ» platform.
- Task
- Text Ranking
- Type
- xlm-roberta
- License
- mit
- Created
- Jun 20, 2026
- Updated
- Jun 20, 2026