bangla-embed-e5-small-banglish

Bangla-Embed-E5-Banglish

A compact 118M Bengali sentence encoder that additionally supports romanized Bengali (Banglish) and, uniquely, retrieves across scripts — aligning romanized queries with their native-Bengali and English counterparts. Rebased from intfloat/multilingual-e5-small by three-stage distillation from a BGE-M3 teacher, with IndicXlit-romanized views added in the distillation and contrastive stages.

Highlights

• Cross-script retrieval: on a held-out, human-typed romanization set (disjoint from the training transliterator) it retrieves native counterparts at acc@1 0.85 / 0.96, versus ≤0.27 for all baselines, and is the only model that retrieves across script in a controlled product-search study (MRR@10 0.31 vs ≤0.11).
• MTEB(Indic) Bengali mean 0.698 — statistically tied with the 4.8× larger BGE-M3 (0.700) and mE5-large (0.690); strongest encoder in the ≤120M tier.

For native-Bengali-only use, the companion kazalbrur/bangla-embed-e5-small is slightly stronger on some full-corpus retrieval sets.

Comparison — MTEB(Indic) Bengali subset

Main score per task type (mteb 2.12), single shared harness. This model is bolded.

Retr=BelebeleRetrieval, Class=BengaliSentiment, Bitext-G/C=IN22 Gen/Conv, Rerank=WikipediaReranking, STS=IndicCrosslingualSTS, Clust=SIB200. This model ranks second on mean and tops cross-lingual STS; the top-3 means span only 0.010 and are statistically indistinguishable under a 7-task paired bootstrap (treat sub-0.02 gaps as ties), i.e. on par with the 4.8× larger BGE-M3.

Cross-script (Banglish) retrieval — unique to this model

acc@1 on human-typed, held-out romanized queries retrieving native counterparts (disjoint from the training transliterator):

Only this model aligns romanized Bengali with native script/English.

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("kazalbrur/bangla-embed-e5-small-banglish")
# romanized (Banglish) query -> native Bengali passage
q = model.encode(["dhaka kothay?"], prompt_name="query", normalize_embeddings=True)
d = model.encode(["ঢাকা বাংলাদেশের রাজধানী।"], prompt_name="passage", normalize_embeddings=True)
print((q @ d.T)[0, 0])

E5-family conventions: prefix queries with query: and passages with passage: (handled by prompt_name). Output dimension is 1024, L2-normalized.

Training & data

Three-stage curriculum (AdamW, cosine schedule, bf16). Distillation over ~18.7M EN–BN parallel pairs plus ~1M IndicXlit-romanized views; supervised contrastive fine-tuning (MNR) on ~2.77M pairs (Bangla-native core + SWIM-IR + MS MARCO-bn + ~0.32M romanized views); NLI polish (XNLI-bn, IndicXNLI-bn, MNLI-en). Released under the MIT license. Note that some training sources carry their own (in some cases non-commercial) terms — verify upstream data terms before commercial deployment.

Limitations

Training romanization is synthetic (IndicXlit); generalization is validated on a human-typed held-out set and a rule-based generator, but coverage is Standard Bangla only (regional typing conventions and varieties such as Sylheti/Chittagonian are unrepresented). On full-corpus MIRACL-bn/Mr.TyDi-bn it trails larger models and the untuned backbone; all metrics are automatic.