Downloads · 30 days
42
7% of all-time downloads
hotchpotch/vespa-onnx-BAAI-bge-m3-only-dense
vespa-onnx-BAAI-bge-m3-only-dense is a feature extraction model from hotchpotch. Use it when you need embeddings to search or compare text. It is set up for transformers. The card lists the license as mit.
Converted BAAI/bge-m3 model (dense retriever only) in onnx fp16/int8 format for use with Vespa Embedding.
Downloads · 30 days
42
7% of all-time downloads
All-time downloads
631
Public
Repo size
1.7 GB
Likes
3
Public
Click a slice to open those files.
.onnx1.7 GB · 99%
From the Hugging Face model README
Converted BAAI/bge-m3 model (dense retriever only) in onnx fp16/int8 format for use with Vespa Embedding.
The model was quantized using the optimum toolkit.
Notice: FP16 works well with Vespa versions 8.325.46 and above.
<component id="bge_m3" type="hugging-face-embedder">
<transformer-model
url="https://huggingface.co/hotchpotch/vespa-onnx-BAAI-bge-m3-only-dense/resolve/main/BAAI-bge-m3_fp16.onnx" />
<!-- or int8 quantization model
<transformer-model
url="https://huggingface.co/hotchpotch/vespa-onnx-BAAI-bge-m3-only-dense/resolve/main/BAAI-bge-m3_quantized.onnx"
/>
-->
<tokenizer-model
url="https://huggingface.co/hotchpotch/vespa-onnx-BAAI-bge-m3-only-dense/resolve/main/tokenizer.json" />
<normalize>true</normalize>
<pooling-strategy>cls</pooling-strategy>
</component>
# FP16 model has a larger file size, which can result in longer deployment times.
vespa deploy --wait 1800 .
# https://github.com/vespa-engine/sample-apps/blob/master/simple-semantic-search/export_hf_model_from_hf.py
./export_hf_model_from_hf.py --hf_model BAAI/bge-m3 --output_dir bge-m3
optimum-cli onnxruntime quantize --onnx_model ./bge-m3 -o bge-m3-large_quantized --avx512_vnni
# https://github.com/vespa-engine/sample-apps/blob/master/simple-semantic-search/export_hf_model_from_hf.py
./export_hf_model_from_hf.py --hf_model BAAI/bge-m3 --output_dir bge-m3
# https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/float16.py
import onnx
from onnxruntime.transformers.float16 import convert_float_to_float16
onnx_model = onnx.load("bge-m3/BAAI-bge-m3.onnx")
model_fp16 = convert_float_to_float16(onnx_model, disable_shape_infer=True)
onnx.save(model_fp16, "bge-m3/BAAI-bge-m3_fp16.onnx")
The license for this model is based on the original license (found in the LICENSE file in the project's root directory), which is the MIT License.
All credits for this model go to the authors of BAAI/bge-m3 and the associated researchers and organizations. When using this model, please be sure to attribute the original authors.