Downloads · 30 days
15
26% of all-time downloads
monju-lab/ruri-v3-130m-int8-onnx
ruri-v3-130m-int8-onnx is a feature extraction model from monju-lab. Use it when you need embeddings to search or compare text. It is set up for transformers. The card lists the license as apache-2.0.
ONNX export of cl-nagoya/ruri-v3-130m (132.1M params, ModernBERT-ja architecture fine-tuned for Japanese sentence embeddings), converted for the feature-extraction task and dynamically quantized to INT8.
Downloads · 30 days
15
26% of all-time downloads
All-time downloads
58
Public
Repo size
135 MB
Likes
0
Public
Click a slice to open those files.
.onnx133 MB · 94%
From the Hugging Face model README
ONNX export of cl-nagoya/ruri-v3-130m (132.1M params, ModernBERT-ja architecture fine-tuned for Japanese sentence embeddings), converted for the feature-extraction task and dynamically quantized to INT8.
| File | Precision | Size | Use case |
|---|---|---|---|
onnx/model_quantized.onnx | int8 (dynamic) | 127 MB (~4x smaller than the 505 MB original) | Recommended for CPU serving |
An fp32 ONNX reference (505 MB, zero measured quality loss, ~3x faster than PyTorch) was also produced during conversion but isn't included in this upload — ask if you want it added.
Ruri v3 uses a 1+3 prefix scheme — prepend one of these to your text before embedding, depending on the use case:
| Prefix | Use case |
|---|---|
| (none) | General semantic similarity |
トピック: | Classification / clustering / topic-level info |
検索クエリ: | Retrieval queries |
検索文書: | Documents to be retrieved |
Pooling is mean pooling over last_hidden_state (with attention mask), matching the original Sentence-Transformers 1_Pooling config included in this repo. Output dimensionality is 512.
Measured on a CPU-only sandbox (onnxruntime CPUExecutionProvider), single-sequence inference, over the same 20-sentence Japanese set used for modernbert-ja-130m (no prefix, general semantic similarity setting). Quality is reported against the original PyTorch fp32 model both as mean cosine similarity of individual embeddings and as Spearman rank correlation of the full pairwise similarity matrix.
| Variant | Latency (mean, batch=1) | vs. PyTorch | File size | Cosine sim to original | Similarity-ranking correlation (Spearman ρ) |
|---|---|---|---|---|---|
| PyTorch fp32 (original) | 85.9 ms | 1.0x | 505 MB (safetensors) | 1.0000 | 1.0000 |
| ONNX fp32 | 27.2 ms | 3.2x faster | 505 MB | 1.0000 | 1.0000 |
| ONNX fp16 | 28.7 ms | 3.0x faster* | 252 MB | 0.999999 | 0.99999 |
| ONNX int8 (dynamic) | 12.3 ms | 7.0x faster | 127 MB (4x smaller) | 0.9890 (min 0.9770) | 0.9447 |
* fp16 shows no real latency benefit here because this is a CPU-only benchmark; fp16 is meant for GPU tensor cores. Not included in this upload.
Recommendation: onnx/model_quantized.onnx (int8) gives the best combination of size and CPU latency, at the cost of some embedding precision (~0.99 cosine similarity, 0.94 ranking correlation vs. the original). If your downstream task is sensitive to fine-grained similarity ranking (e.g. re-ranking, nearest-neighbor retrieval with tight margins), validate against your own eval set before relying on it, or ask for the fp32 ONNX variant instead (same latency-class win as modernbert-ja-130m's fp32 export, with zero measured quality loss).
import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("<this-repo>")
session = ort.InferenceSession("onnx/model_quantized.onnx", providers=["CPUExecutionProvider"])
def embed(sentences, prefix=""):
texts = [prefix + s for s in sentences]
enc = tokenizer(texts, padding=True, return_tensors="np")
last_hidden_state, = session.run(
None, {"input_ids": enc["input_ids"], "attention_mask": enc["attention_mask"]}
)
mask = enc["attention_mask"][..., None].astype(np.float32)
summed = (last_hidden_state.astype(np.float32) * mask).sum(axis=1)
counts = np.clip(mask.sum(axis=1), 1e-9, None)
return summed / counts
queries = embed(["東京の人口は?"], prefix="検索クエリ: ")
docs = embed(["東京都の人口は約1400万人です。"], prefix="検索文書: ")
cos_sim = (queries[0] @ docs[0]) / (np.linalg.norm(queries[0]) * np.linalg.norm(docs[0]))
print(cos_sim)
optimum (optimum.exporters.onnx), task feature-extraction, opset 18.onnxruntime.quantization.quantize_dynamic (QInt8 weights).modules.json, config_sentence_transformers.json, sentence_bert_config.json, 1_Pooling/config.json) are carried over from the original repo for reference/compatibility, though this ONNX export is loaded via onnxruntime / optimum directly rather than the sentence_transformers library.Base model card, license (Apache-2.0), training details, and MTEB scores: cl-nagoya/ruri-v3-130m.