Downloads · 30 days
142
5% of all-time downloads
qilowoq/mmarco-mMiniLMv2-L12-H384-v1-en-ru
mmarco-mMiniLMv2-L12-H384-v1-en-ru is a text classification model from qilowoq. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.
This is a truncated version of jeffwan/mmarco-mMiniLMv2-L12-H384-v1.
Downloads · 30 days
142
5% of all-time downloads
All-time downloads
2.9K
Public
Parameters
44.8M
179 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors179 MB · 98%
From the Hugging Face model README
This is a truncated version of jeffwan/mmarco-mMiniLMv2-L12-H384-v1.
This model has only English and Russian tokens left in the vocabulary. Thus making it twice as small as the original model while producing the same embeddings.
This model was trained on the MMARCO dataset. It is a machine translated version of MS MARCO using Google Translate. It was translated to 14 languages. In our experiments, we observed that it performs also well for other languages.
As a base model, we used the multilingual MiniLMv2 model.
The model can be used for Information Retrieval: Given a query, encode the query will all possible passages (e.g. retrieved with ElasticSearch). Then sort the passages in a decreasing order. See SBERT.net Retrieve & Re-rank for more details. The training code is available here: SBERT.net Training MS Marco
The usage becomes easy when you have SentenceTransformers installed. Then, you can use the pre-trained models like this:
from sentence_transformers import CrossEncoder
model = CrossEncoder('model_name')
scores = model.predict([('Query', 'Paragraph1'), ('Query', 'Paragraph2') , ('Query', 'Paragraph3')])
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model = AutoModelForSequenceClassification.from_pretrained('model_name')
tokenizer = AutoTokenizer.from_pretrained('model_name')
features = tokenizer(['How many people live in Berlin?', 'How many people live in Berlin?'], ['Berlin has a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.', 'New York City is famous for the Metropolitan Museum of Art.'], padding=True, truncation=True, return_tensors="pt")
model.eval()
with torch.no_grad():
scores = model(**features).logits
print(scores)
The model has been truncated in this notebook.