Downloads · 30 days
16
11% of all-time downloads
kzzalews/Nemotron-3-Embed-1B-BF16-onnx
Nemotron-3-Embed-1B-BF16-onnx is a feature extraction model from kzzalews. Use it when you need embeddings to search or compare text. It is set up for sentence-transformers. The card lists the license as apache-2.0.
ONNX export of nvidia/Nemotron-3-Embed-1B-BF16 for use with fastembed-rs and onnxruntime.
Downloads · 30 days
16
11% of all-time downloads
All-time downloads
140
Public
Repo size
7.4 GB
Likes
1
Public
Click a slice to open those files.
.data7.4 GB · 100%
From the Hugging Face model README
ONNX export of nvidia/Nemotron-3-Embed-1B-BF16 for use with fastembed-rs and onnxruntime.
Released 2026-07-16. #1 on RTEB (Retrieval Text Embedding Benchmark) at 1B parameter scale — 27% improvement over the previous Nemotron embedding model.
| Property | Value |
|---|---|
| Base model | nvidia/Nemotron-3-Embed-1B-BF16 |
| Architecture | Ministral3 (no trust_remote_code required) |
| Parameters | ~1B |
| Embedding dimension | 2048 |
| Pooling | Mean pooling over token hidden states |
| Export dtype | float32 (default) / float16 (fp16/ subfolder) |
| Opset | 17 |
| Path | Size | Description |
|---|---|---|
model.onnx | ~2 MB | ONNX graph (float32) |
model.onnx.data | ~4.25 GB | External weights (float32) |
fp16/model.onnx | ~2 MB | ONNX graph (float16) |
fp16/model.onnx.data | ~2.63 GB | External weights (float16) |
tokenizer.json | 16 MB | Tokenizer (Mistral BPE) |
import onnxruntime as ort
import numpy as np
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("kzzalews/Nemotron-3-Embed-1B-BF16-onnx")
sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])
def embed(texts: list[str]) -> np.ndarray:
enc = tokenizer(texts, return_tensors="np", padding=True, truncation=True)
hidden = sess.run(["last_hidden_state"], {
"input_ids": enc["input_ids"],
"attention_mask": enc["attention_mask"],
})[0]
mask = enc["attention_mask"][:, :, np.newaxis].astype(np.float32)
return (hidden * mask).sum(axis=1) / mask.sum(axis=1)
# query prefix required for retrieval
embeddings = embed(["query: How does Kubernetes handle OOMKill events?"])
print(embeddings.shape) # (1, 2048)
float32 variant:
// src/models/text_embedding.rs
ModelInfo {
model: EmbeddingModel::NvidiaNemotron3Embed1BBF16,
dim: 2048,
description: String::from("NVIDIA Nemotron-3 Embed 1B BF16 — #1 RTEB, float32"),
model_code: String::from("kzzalews/Nemotron-3-Embed-1B-BF16-onnx"),
model_file: String::from("model.onnx"),
additional_files: vec!["model.onnx.data".to_string()],
output_key: None,
},
// src/text_embedding/impl.rs
EmbeddingModel::NvidiaNemotron3Embed1BBF16 => Some(Pooling::Mean),
float16 variant (2.63 GB, identical quality, same CPU speed):
ModelInfo {
model: EmbeddingModel::NvidiaNemotron3Embed1BBF16Fp16,
dim: 2048,
description: String::from("NVIDIA Nemotron-3 Embed 1B BF16 — #1 RTEB, float16"),
model_code: String::from("kzzalews/Nemotron-3-Embed-1B-BF16-onnx"),
model_file: String::from("fp16/model.onnx"),
additional_files: vec!["fp16/model.onnx.data".to_string()],
output_key: None,
},
EmbeddingModel::NvidiaNemotron3Embed1BBF16Fp16 => Some(Pooling::Mean),
| Variant | Path | Size | Quality |
|---|---|---|---|
| float32 (default) | model.onnx + model.onnx.data | 4.25 GB | max_diff vs PyTorch = 1.14e-04 |
| float16 | fp16/model.onnx + fp16/model.onnx.data | 2.63 GB | cos_sim = 1.000000 vs float32 |
Note: INT8 dynamic quantization was tested on the predecessor model and rejected (cos_sim = 0.65–0.70 without calibration — unusable for retrieval). Float16 saves ~40% space but does not improve CPU inference speed (CPU emulates fp16 without AVX-512 FP16 hardware).
| Variant | Short text (~11 tokens) |
|---|---|
| float32 | ~129ms |
| float16 | ~128ms (CPU emulation — no speedup without AVX-512 FP16) |
Numerical validation (ONNX float32 vs PyTorch): max diff = 1.14e-04, mean diff ≈ 1e-05.
This model uses instruction prefixes for retrieval:
"query: <text>""passage: <text>"