Llama-Karmx-Indic-Embedding-bge-m3

Llama-Karmx-Indic-Embedding-bge-m3

An Indic embedding model for search and RAG in Hindi, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada, Malayalam, Punjabi and Odia. It is BAAI/bge-m3 fine-tuned for Indian languages. It is a drop-in replacement with the same size, speed, 1,024-dimensional vectors and Sentence Transformers API, and it retrieves better in every one of the ten languages.

• +2.05 nDCG@10 over bge-m3 on IndicQARetrieval (73.77 vs 71.72), with gains in all 10 languages
• +0.85 nDCG@10 over bge-m3 on Belebele retrieval (89.85 vs 88.99), with no language worse
• No query prefix or instruction needed; normalized dense vectors for cosine / dot-product search
• ONNX versions for ONNX Runtime and transformers.js, including a 570 MB int8 file that still beats bge-m3

The quickstart runs semantic search over real passages in all ten languages, then retrieve-and-rerank.

Quick start

Works with Sentence Transformers 3.0 or later (verified identical embeddings on 3.0.1, 5.1.0 and 6.1.0).

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("karmx/Llama-Karmx-Indic-Embedding-bge-m3")
model.max_seq_length = 512

queries = ["भारत की राजधानी क्या है?"]                      # Hindi query
passages = ["नई दिल्ली भारत की राजधानी है।",               # relevant
            "मुंबई महाराष्ट्र की राजधानी है।"]              # not relevant
q = model.encode(queries, normalize_embeddings=True)
p = model.encode(passages, normalize_embeddings=True)
print(q @ p.T)  # higher = more relevant

LangChain:

from langchain_huggingface import HuggingFaceEmbeddings

emb = HuggingFaceEmbeddings(model_name="karmx/Llama-Karmx-Indic-Embedding-bge-m3",
                            encode_kwargs={"normalize_embeddings": True})

ONNX: ONNX Runtime and transformers.js

The onnx/ folder has two exports with inputs input_ids and attention_mask and output last_hidden_state. Take the CLS token (position 0) and L2-normalize it:

The int8 file uses per-channel dynamic quantization. It is 4× smaller than the PyTorch weights and still beats bge-m3 (0.8899) on Belebele.

transformers.js (browser or Node):

import { pipeline } from "@huggingface/transformers";

const extractor = await pipeline("feature-extraction", "karmx/Llama-Karmx-Indic-Embedding-bge-m3", { dtype: "q8" });
const embeddings = await extractor(["भारत की राजधानी क्या है?", "नई दिल्ली भारत की राजधानी है।"],
                                   { pooling: "cls", normalize: true });

ONNX Runtime (Python):

import numpy as np, onnxruntime as ort
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

repo = "karmx/Llama-Karmx-Indic-Embedding-bge-m3"
tokenizer = AutoTokenizer.from_pretrained(repo)
session = ort.InferenceSession(hf_hub_download(repo, "onnx/model_quantized.onnx"))
enc = tokenizer(["भारत की राजधानी क्या है?"], padding=True, truncation=True, max_length=512, return_tensors="np")
cls = session.run(["last_hidden_state"], {"input_ids": enc["input_ids"], "attention_mask": enc["attention_mask"]})[0][:, 0]
embeddings = cls / np.linalg.norm(cls, axis=1, keepdims=True)

For best end-to-end quality, rerank the top 50 results with BAAI/bge-reranker-v2-m3. This embedder raises that two-stage pipeline from 80.41 to 80.76 nDCG@10 on IndicQARetrieval.

Results

Both benchmarks were held out from training and model selection. Scores are nDCG@10 (pytrec_eval) on the complete official corpora. The baseline is bge-m3 dense under the identical protocol (512 tokens, exact top-100 search, no prompts). Intervals come from a paired bootstrap over queries within each language (10,000 samples).

IndicQARetrieval (MTEB, 16,775 queries):

MRR rises from 0.6768 to 0.6992 and recall@100 from 0.9802 to 0.9828.

Belebele retrieval (facebook/belebele, question → passage, 9,000 human-translated questions):

How it was trained

The stronger cross-encoder BAAI/bge-reranker-v2-m3 teaches bge-m3 how to rank its own top candidates (listwise knowledge distillation):

• Loss: KL divergence between the reranker's softmax over each query's candidates and the embedder's softmax over cosine similarities / 0.02. The candidates are the labeled passage plus bge-m3's 3 highest-ranked passages. An in-batch InfoNCE term is added, with same-article passages masked.
• Data: 28,502 questions over Wikipedia paragraphs in the ten languages (ai4bharat/Indic-Rag-Suite, article-level train split) plus MIRACL training judgments for Hindi, Bengali and Telugu. Benchmark questions, passages and source articles were removed first.
• Setup: full fine-tuning with frozen word embeddings, lr 1e-5 (5% warmup, linear decay), 32 queries per batch, BF16, one pass (885 steps, about 21 minutes on one RTX 5090). The checkpoint was chosen on a 5,000-query article-held-out dev split, where nDCG@10 rose from 0.8154 to 0.8454.

Limitations

• Dense retrieval only. bge-m3's sparse and multi-vector (ColBERT) heads are not included.
• Tuned for the ten listed languages; English and other languages were not evaluated.
• Default max_seq_length is 512 tokens, the evaluated setting. The architecture accepts up to 8,192, but longer inputs were not evaluated.
• Single training run (one seed).
• No claim for romanized Hindi or Hindi–English code-mixed text.

License and attribution

MIT, like the base model. Built with Llama: the training questions in Indic-Rag-Suite were generated by Meta-Llama-3.3-70B-Instruct. The Llama 3.3 Community License requires models trained on such outputs to carry a name beginning with "Llama". Passages are Wikipedia text (CC BY-SA 4.0). MIRACL is Apache-2.0. The base model bge-m3 is MIT, and the teacher bge-reranker-v2-m3 is Apache-2.0, both by BAAI.

Citation

@misc{karmx2026indicembedding,
  title  = {Llama-Karmx-Indic-Embedding-bge-m3: Reranker-Distilled BGE-M3 for Ten Indic Languages},
  author = {Karmx},
  year   = {2026},
  url    = {https://huggingface.co/karmx/Llama-Karmx-Indic-Embedding-bge-m3}
}