NeoMME-800M-Retriever-ST-dense

<p align="left"> <img src="https://github.com/tonywu71/colpali-cookbooks/blob/6ef1332da6bcb48c7ef1f19b25bfa555be7031a8/assets/neomme/neomme_logo.webp?raw=true" alt="NeoMME logo" style="max-height: 140px;"> </p>

NeoMME-Retriever (800M): Single-Tower Multimodal-Native Multilingual Foundation Encoder 🔎

[!IMPORTANT] NeoMME-Retriever (800M) variants:

• Default (transformers): Returns dense and multi-vector embeddings together with a single forward pass. Recommended for most use cases and inference.
• ST dense [current]: Supports independent dense fine-tuning with Sentence Transformers.
• ST late-interaction: Supports independent multi-vector fine-tuning with Sentence Transformers.

NeoMME-800M-Retriever-ST-dense is a model for multimodal document retrieval. Fine-tuned from NeoMME-800M, it encodes text queries and documents (text or page screenshots) using one shared bidirectional Transformer encoder.

This model can be used with Sentence Transformers, but can only generate dense embeddings.

<table> <thead> <tr style="background-color: rgba(146, 81, 247, 0.20);"><th>Specification</th><th>Value</th></tr> </thead> <tbody> <tr><td>Parameters</td><td>800M</td></tr> <tr><td>Vocabulary</td><td>131,072 tokens</td></tr> <tr><td>Context length</td><td>16,384 tokens</td></tr> <tr><td>Hidden size</td><td>1,792</td></tr> <tr><td>Image patches</td><td>32 × 32 pixels, up to 2,048 pixels on the longest side (default)</td></tr> <tr><td>Dense embeddings</td><td>1,792 dimensions (Matryoshka: [128, 256, 512, 1,024, 1,792])</td></tr> <tr><td>Dense pooling strategy</td><td>Mean</td></tr> </tbody> </table>

Dense embeddings are L2-normalized and use cosine similarity. They match NeoMMEForRetrieval.dense_embeddings.

Performance

All scores use the metric shown at the full trained dimensions. Higher is better. ViDoRe v3, v2, and v1 measure visual document retrieval, while BEIR-15 measures text retrieval.

<table> <thead> <tr><th rowspan="2">Benchmark</th><th rowspan="2">Metric</th><th colspan="2">NeoMME-260M</th><th colspan="2" style="background-color: rgba(37, 99, 235, 0.20);">NeoMME-800M</th></tr> <tr><th>Late interaction</th><th>Dense</th><th>Late interaction</th><th style="background-color: rgba(37, 99, 235, 0.20);">Dense [current]</th></tr> </thead> <tbody> <tr><td>ViDoRe v3</td><td>nDCG@10</td><td>0.5226</td><td>0.3907</td><td>0.5560</td><td style="background-color: rgba(37, 99, 235, 0.20);">0.4391</td></tr> <tr><td>ViDoRe v2</td><td>nDCG@5</td><td>0.5218</td><td>0.4075</td><td>0.5591</td><td style="background-color: rgba(37, 99, 235, 0.20);">0.4475</td></tr> <tr><td>ViDoRe v1</td><td>nDCG@5</td><td>0.8598</td><td>0.7552</td><td>0.8744</td><td style="background-color: rgba(37, 99, 235, 0.20);">0.7993</td></tr> <tr><td>BEIR-15</td><td>nDCG@10</td><td>0.4881</td><td>0.3055</td><td>0.5126</td><td style="background-color: rgba(37, 99, 235, 0.20);">0.3686</td></tr> </tbody> </table>

Usage

pip install -U "sentence-transformers[image]"

from sentence_transformers import SentenceTransformer


model = SentenceTransformer("Hcompany/NeoMME-800M-Retriever-ST-dense")

queries = [
    "Quelle partie de la production pétrolière du Kazakhstan provient de champs en mer ?",
    "Which hour of the day had the highest overall electricity generation in 2019?",
]
documents = [
    "https://github.com/tonywu71/colpali-cookbooks/blob/main/examples/data/shift_kazakhstan.jpg?raw=true",
    "https://github.com/tonywu71/colpali-cookbooks/blob/main/examples/data/energy_electricity_generation.jpg?raw=true",
]

query_embeddings = model.encode_query(queries, convert_to_tensor=True)
document_embeddings = model.encode_document(documents, convert_to_tensor=True)
scores = model.similarity(query_embeddings, document_embeddings)

# Expected: scores[0, 0] > scores[0, 1] and scores[1, 1] > scores[1, 0].
print(scores)

The score tensor has shape (num_queries, num_documents) and scores[i, j] is the score between query i and document j. A larger value indicates a closer match.

Training

NeoMME-800M-Retriever was fine-tuned from NeoMME-800M on text retrieval and document-page images. Training uses a joint late-interaction and Matryoshka dense contrastive objective.

The NeoMME technical report describes the full fine-tuning recipe.

Limitations

With Sentence Transformers, only one of the two retrieval heads can be used at a time.

License

Model weights are released under the Apache 2.0 license.

Citation

@misc{lac2026neommesingletowermultimodalnativemultilingual,
      title={NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference},
      author={Aurélien Lac and Tony Wu},
      year={2026},
      eprint={2609.01657},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2609.01657},
}