EmbeddingGemma-300M-Amharic-fp32

EmbeddingGemma-300M-Amharic (fp32)

Google EmbeddingGemma-300M fine-tuned on Amharic passage retrieval supervision (full-precision fp32 checkpoint). For the BF16-optimized variant see kiyam/EmbeddingGemma-300M-Amharic.

Published alongside the ACL 2026 MeLLM Workshop paper:

The Multilingual Curse at the Retrieval Layer: Evidence from Amharic
Yosef Worku Alemneh, Kidist Amde Mekonnen, Maarten de Rijke
The 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM), ACL 2026

Results on Amharic Passage Retrieval Dataset V2

Evaluation dataset: rasyosef/Amharic-Passage-Retrieval-Dataset-V2

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("kiyam/EmbeddingGemma-300M-Amharic-fp32")

queries = ["የኢትዮጵያ ዋና ከተማ የትኛው ናት?"]
passages = ["አዲስ አበባ የኢትዮጵያ ዋና ከተማ ናት።"]

query_embeddings = model.encode(queries, normalize_embeddings=True)
passage_embeddings = model.encode(passages, normalize_embeddings=True)

scores = query_embeddings @ passage_embeddings.T

Training Details

• Base model: google/embeddinggemma-300m (300M parameters)
• Training data: yosefw/amharic-news-retrieval-dataset-v2-with-negatives-V2
• Objective: MultipleNegativesRankingLoss + MatryoshkaLoss (dims: 768, 256)
• Epochs: 6 | Batch size: 32 | Grad. accum.: 4
• Learning rate: 4e-5, cosine schedule
• Max sequence length: 512

Citation

@inproceedings{alemneh2026amharicir,
  title     = {The Multilingual Curse at the Retrieval Layer: Evidence from Amharic},
  author    = {Alemneh, Yosef Worku and Mekonnen, Kidist Amde and de Rijke, Maarten},
  booktitle = {Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM), ACL 2026},
  year      = {2026},
}