Downloads · 30 days
20
3% of all-time downloads
optimum/bge-base-en-v1.5-neuronx-bs-1
bge-base-en-v1.5-neuronx-bs-1 is a feature extraction model from optimum. Use it when you need embeddings to search or compare text. It is set up for transformers. The card lists the license as apache-2.0.
This model is compiled for neuronx devices (eg. on inf2 instance).
Downloads · 30 days
20
3% of all-time downloads
All-time downloads
618
Public
Repo size
2.4 GB
Likes
0
Public
Click a slice to open those files.
.neuron197 MB · 100%
From the Hugging Face model README
This model is compiled for neuronx devices (eg. on inf2 instance).
This original checkpoint is BAAI/bge-base-en-v1.5.
Here below is the command used for exporting this model:
optimum-cli export neuron -m BAAI/bge-base-en-v1.5 --sequence_length 384 --batch_size 1 --task feature-extraction bge_emb/
To use the compiled artifacts for inference, here is an example:
from transformers import AutoTokenizer
from optimum.neuron import NeuronModelForSenetenceTransformers
emb_model = NeuronModelForSenetenceTransformers.from_pretrained("optimum/bge-base-en-v1.5-neuronx")
inputs = tokenizer("Hamilton is considered to be the best musical of human history.", return_tensors="pt")
emb = emb_model(**inputs)
# ["token_embeddings", "sentence_embedding"]