Downloads · 30 days
8
5% of all-time downloads
drexalt/splade-NeoBERT-RetroMAE-pretrain-lighton-msmarco-triplets-20250919-064009
splade-NeoBERT-RetroMAE-pretrain-lighton-msmarco-triplets-20250919-064009 is a feature extraction model from drexalt. Use it when you need embeddings to search or compare text. It is set up for sentence-transformers. The card lists the license as apache-2.0.
This is a SPLADE Sparse Encoder model finetuned from drexalt/NeoBERT-RetroMAE-pretrain on the ms-marco-en-bge-gemma dataset using the sentence-transformers library. It maps sentences & paragraphs to a 30522-dimensiona…
Downloads · 30 days
8
5% of all-time downloads
All-time downloads
163
Public
Parameters
245M
981 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors981 MB · 100%
From the Hugging Face model README
This is a SPLADE Sparse Encoder model finetuned from drexalt/NeoBERT-RetroMAE-pretrain on the ms-marco-en-bge-gemma dataset using the sentence-transformers library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
SparseEncoder(
(0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'NeoBERTLMHead'})
(1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SparseEncoder
# Download from the 🤗 Hub
model = SparseEncoder("drexalt/splade-NeoBERT-RetroMAE-pretrain-lighton-msmarco-triplets-20250919-064009")
# Run inference
sentences = [
'The weather is lovely today.',
"It's so sunny outside!",
'He drove to the stadium.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 30522]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
NanoSciFact, NanoMSMARCO, NanoTouche2020, NanoSCIDOCS, NanoNFCorpus, NanoClimateFEVER, NanoDBPedia, NanoFEVER, NanoFiQA2018, NanoHotpotQA, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoQuoraRetrieval, NanoSCIDOCS, NanoArguAna, NanoSciFact and NanoTouche2020| Metric | NanoSciFact | NanoMSMARCO | NanoTouche2020 | NanoSCIDOCS | NanoNFCorpus | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoNQ | NanoQuoraRetrieval | NanoArguAna |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| dot_accuracy@1 | 0.54 | 0.46 | 0.7143 | 0.36 | 0.44 | 0.26 | 0.78 | 0.78 | 0.42 | 0.86 | 0.48 | 0.88 | 0.12 |
| dot_accuracy@3 | 0.68 | 0.64 | 0.9184 | 0.56 | 0.6 | 0.46 | 0.92 | 0.9 | 0.54 | 0.92 | 0.68 | 0.98 | 0.44 |
| dot_accuracy@5 | 0.74 | 0.7 | 0.9592 | 0.66 | 0.64 | 0.52 | 0.94 | 0.94 | 0.58 | 0.94 | 0.74 | 0.98 | 0.54 |
| dot_accuracy@10 | 0.8 | 0.76 | 0.9796 | 0.8 | 0.64 | 0.62 | 0.94 | 1.0 | 0.68 | 0.94 | 0.82 | 0.98 | 0.64 |
| dot_precision@1 | 0.54 | 0.46 | 0.7143 | 0.36 | 0.44 | 0.26 | 0.78 | 0.78 | 0.42 | 0.86 | 0.48 | 0.88 | 0.12 |
| dot_precision@3 | 0.2467 | 0.2133 | 0.619 | 0.26 | 0.4 | 0.16 | 0.6133 | 0.32 | 0.26 | 0.44 | 0.2267 | 0.38 | 0.1467 |
| dot_precision@5 | 0.168 | 0.14 | 0.6163 | 0.216 | 0.34 | 0.116 | 0.588 | 0.2 | 0.176 | 0.296 | 0.156 | 0.244 | 0.108 |
| dot_precision@10 | 0.09 | 0.076 | 0.4898 | 0.146 | 0.272 | 0.074 | 0.502 | 0.106 | 0.108 | 0.156 | 0.086 | 0.13 | 0.064 |
| dot_recall@1 | 0.515 | 0.46 | 0.049 | 0.0757 | 0.0621 | 0.1433 | 0.1039 | 0.7167 | 0.2234 | 0.43 | 0.46 | 0.7873 | 0.12 |
| dot_recall@3 | 0.655 | 0.64 | 0.1284 | 0.1617 | 0.0997 | 0.2283 | 0.1658 | 0.8633 | 0.3617 | 0.66 | 0.63 | 0.9187 | 0.44 |
| dot_recall@5 | 0.73 | 0.7 | 0.202 | 0.2217 | 0.1159 | 0.26 | 0.2364 | 0.9033 | 0.3872 | 0.74 | 0.7 | 0.9453 | 0.54 |
| dot_recall@10 | 0.79 | 0.76 | 0.3157 | 0.2997 | 0.1399 | 0.3007 | 0.338 | 0.9633 | 0.4999 | 0.78 | 0.78 | 0.966 | 0.64 |
| dot_ndcg@10 | 0.656 | 0.603 | 0.5537 | 0.2955 | 0.3541 | 0.2727 | 0.6253 | 0.8531 | 0.4203 | 0.7586 | 0.6249 | 0.9174 | 0.3757 |
| dot_mrr@10 | 0.6174 | 0.5534 | 0.8163 | 0.4941 | 0.52 | 0.3772 | 0.8407 | 0.8478 | 0.4946 | 0.8907 | 0.5917 | 0.9133 | 0.2914 |
| dot_map@100 | 0.6162 | 0.5666 | 0.401 | 0.2172 | 0.1637 | 0.2221 | 0.4622 | 0.8092 | 0.3593 | 0.698 | 0.5718 | 0.8929 | 0.3018 |
| query_active_dims | 215.0 | 99.8 | 93.6939 | 152.1 | 91.0 | 233.42 | 101.42 | 147.24 | 104.18 | 138.0 | 97.18 | 98.04 | 462.06 |
| query_sparsity_ratio | 0.993 | 0.9967 | 0.9969 | 0.995 | 0.997 | 0.9924 | 0.9967 | 0.9952 | 0.9966 | 0.9955 | 0.9968 | 0.9968 | 0.9849 |
| corpus_active_dims | 529.1915 | 296.6851 | 370.4649 | 498.6517 | 497.9546 | 401.8574 | 330.4877 | 403.6111 | 358.2927 | 333.8576 | 359.4038 | 103.3716 | 448.9967 |
| corpus_sparsity_ratio | 0.9827 | 0.9903 | 0.9879 | 0.9837 | 0.9837 | 0.9868 | 0.9892 | 0.9868 | 0.9883 | 0.9891 | 0.9882 | 0.9966 | 0.9853 |
NanoBEIR_mean{
"dataset_names": [
"scifact",
"msmarco",
"touche2020",
"scidocs",
"nfcorpus"
]
}
| Metric | Value |
|---|---|
| dot_accuracy@1 | 0.5029 |
| dot_accuracy@3 | 0.6757 |
| dot_accuracy@5 | 0.7398 |
| dot_accuracy@10 | 0.7918 |
| dot_precision@1 | 0.5029 |
| dot_precision@3 | 0.3465 |
| dot_precision@5 | 0.2977 |
| dot_precision@10 | 0.2143 |
| dot_recall@1 | 0.2324 |
| dot_recall@3 | 0.3329 |
| dot_recall@5 | 0.3947 |
| dot_recall@10 | 0.4602 |
| dot_ndcg@10 | 0.4922 |
| dot_mrr@10 | 0.5998 |
| dot_map@100 | 0.3929 |
| query_active_dims | 129.8795 |
| query_sparsity_ratio | 0.9957 |
| corpus_active_dims | 407.1837 |
| corpus_sparsity_ratio | 0.9867 |
NanoBEIR_mean{
"dataset_names": [
"climatefever",
"dbpedia",
"fever",
"fiqa2018",
"hotpotqa",
"msmarco",
"nfcorpus",
"nq",
"quoraretrieval",
"scidocs",
"arguana",
"scifact",
"touche2020"
]
}
| Metric | Value |
|---|---|
| dot_accuracy@1 | 0.5457 |
| dot_accuracy@3 | 0.7106 |
| dot_accuracy@5 | 0.7599 |
| dot_accuracy@10 | 0.8154 |
| dot_precision@1 | 0.5457 |
| dot_precision@3 | 0.3297 |
| dot_precision@5 | 0.2588 |
| dot_precision@10 | 0.1769 |
| dot_recall@1 | 0.319 |
| dot_recall@3 | 0.4579 |
| dot_recall@5 | 0.514 |
| dot_recall@10 | 0.5826 |
| dot_ndcg@10 | 0.5623 |
| dot_mrr@10 | 0.6345 |
| dot_map@100 | 0.4832 |
| query_active_dims | 156.4915 |
| query_sparsity_ratio | 0.9949 |
| corpus_active_dims | 359.4253 |
| corpus_sparsity_ratio | 0.9882 |
| query_id | document_ids | scores | |
|---|---|---|---|
| type | int | list | list |
The model card is truncated. Read the rest on Hugging Face.