Downloads · 30 days
10
13% of all-time downloads
Zeolit/lettuce-minilm-bge-m3-v1-onnx-int8
lettuce-minilm-bge-m3-v1-onnx-int8 is a feature extraction model from Zeolit. Use it when you need embeddings to search or compare text. It is set up for onnx. The card lists the license as apache-2.0.
Lettuce MiniLM BGE-M3 v1 is a distilled and quantized MiniLM-based sentence embedding model designed for fast, on-device semantic search and conversational memory retrieval.
Downloads · 30 days
10
13% of all-time downloads
All-time downloads
75
Public
Repo size
22.8 MB
Likes
0
Public
Click a slice to open those files.
.onnx22.8 MB · 96%
From the Hugging Face model README
Lettuce MiniLM BGE-M3 v1 is a distilled and quantized MiniLM-based sentence embedding model designed for fast, on-device semantic search and conversational memory retrieval.
It provides:
This model is ideal for applications where speed and size matter, such as:
all-MiniLM-L6-v2 (6-layer MiniLM encoder)BAAI/bge-m3This model was trained by distilling a larger teacher embedding model (BGE-M3) into a compact MiniLM student, then exporting and quantizing the model to ONNX int8 for maximum runtime efficiency.
import onnxruntime as ort
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("zeolit/lettuce-minilm-bge-m3-v1-onnx-int8")
session = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])
def embed(texts):
if isinstance(texts, str):
texts = [texts]
enc = tokenizer(
texts,
padding=True,
truncation=True,
max_length=256,
return_tensors="np",
)
outputs = session.run(
["sentence_embedding"],
{
"input_ids": enc["input_ids"],
"attention_mask": enc["attention_mask"]
},
)[0]
return outputs # (batch_size, 384)
embeddings = embed("Sam took a bullet for me.")
print(embeddings.shape) # (1, 384)
This model is released under the Apache-2.0 license.
Derived from:
sentence-transformers/all-MiniLM-L6-v2 — Apache-2.0BAAI/bge-m3 — MIT LicenseThanks to: