Downloads · 30 days
186
32% of all-time downloads
xuanthanh318/bge-m3-int4-asym-g32-openvino
bge-m3-int4-asym-g32-openvino is a feature extraction model from xuanthanh318. Use it when you need embeddings to search or compare text. It is set up for openvino. The card lists the license as mit.
Locally compressed derivative of BAAI/bge-m3 for dense embedding inference with OpenVINO. NNCF settings: INT4ASYM, group size 32, ratio 1.0, scale estimation disabled. NNCF retained protected backup layers as INT8 ASYM.
Downloads · 30 days
186
32% of all-time downloads
All-time downloads
582
Public
Repo size
465 MB
Likes
0
Public
Click a slice to open those files.
.bin443 MB · 95%
From the Hugging Face model README
Locally compressed derivative of BAAI/bge-m3 for dense embedding inference with
OpenVINO. NNCF settings: INT4_ASYM, group size 32, ratio 1.0, scale estimation
disabled. NNCF retained protected backup layers as INT8 ASYM.
LoCoMo (English, dense retrieval) macro average: nDCG@10 45.994%, Recall@10 60.878%, MRR@10 44.845%, Hit@10 70.640%. Tested with OpenVINO 2026.3 on Intel Arc 140V GPU. Embedding dimension is 1024 and maximum sequence length is 8192.
import numpy as np
import openvino as ov
from huggingface_hub import snapshot_download
from transformers import AutoTokenizer
repo_id = "YOUR_USERNAME/bge-m3-int4-asym-g32-openvino"
path = snapshot_download(repo_id)
tokenizer = AutoTokenizer.from_pretrained(path)
compiled = ov.Core().compile_model(
f"{path}/openvino/openvino_model.xml",
"GPU", # use "CPU" when needed
)
tokens = tokenizer(
["xin chào, đây là phép thử"],
padding=True,
truncation=True,
max_length=8192,
return_tensors="np",
)
embedding = np.asarray(
compiled({
"input_ids": tokens["input_ids"],
"attention_mask": tokens["attention_mask"],
})[compiled.output("sentence_embedding")],
dtype=np.float32,
)
embedding /= np.maximum(np.linalg.norm(embedding, axis=1, keepdims=True), 1e-12)
print(embedding.shape) # (1, 1024)
This artifact exposes the dense sentence_embedding output. It does not expose
BGE-M3 sparse or ColBERT retrieval modes. The original model is MIT licensed; see
the BAAI/bge-m3 model card for attribution and citation.