translategemma-12b-khutbah

TranslateGemma-12B-Khutbah

Arabic → German translation model specialized for Islamic sermons (khutbahs): religious rhetoric, classical rhymed prose (saj'), hadith quotations, and Islamic-German terminology (Allah, Gesandter, Gottesfurcht, Diesseits/Jenseits, …).

Fine-tuned from google/translategemma-12b-it for MinbarAI, a live mosque-overlay translator that displays German subtitles for the congregation while the Imam speaks.

Results

Evaluated on a 74-case khutbah benchmark (Quran recitation, liturgical formulas, hadith, and real sermon rhetoric from Grand-Mosque khutbahs), scored as cosine similarity of multilingual sentence embeddings against up to three reference translations (held out from training):

Qualitative: the stock model's strict-template refusals on rhymed prose and hadith fragments are eliminated; Islamic terminology is rendered consistently.

Training

• Data: 24,240 Arabic–German pairs — the Quran against four German editions (Bubenheim & Elyas, Khoury, Zaidan, Abu Rida), canonical liturgical formulas (khutbat al-hajah, shahada, salawat, du'a), the most-quoted khutbah hadiths, and Islamic terminology sentences. All benchmark sentences excluded by normalized-text matching.
• Method: QLoRA (4-bit nf4 base, LoRA r=16, α=32, all attention+MLP projections), lr 1e-4 cosine, 1 epoch, bf16 on A100.
• Curves: train loss 3.28 → 0.27; validation loss monotonically 0.464 → 0.269 (no overfitting).
• Training code: MinbarAI/training.

Files

• Merged fp16 safetensors (transformers-ready)
• model-Q4_K_M.gguf — quantized for llama.cpp / Ollama serving

Usage

The model expects TranslateGemma's structured chat template:

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("Yacinedh/translategemma-12b-khutbah")
model = AutoModelForCausalLM.from_pretrained("Yacinedh/translategemma-12b-khutbah", device_map="auto")

messages = [{"role": "user", "content": [{
    "type": "text", "source_lang_code": "ar", "target_lang_code": "de-DE",
    "text": "أوصيكم ونفسي بتقوى الله عز وجل",
}]}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(inputs, max_new_tokens=128)[0][inputs.shape[1]:], skip_special_tokens=True))

Ollama (from the GGUF): create a Modelfile FROM ./model-Q4_K_M.gguf with the Gemma turn template and PARAMETER stop <end_of_turn> — see MinbarAI/server/modal_app.py for the exact Modelfile used in production.

Intended use & limitations

• Target register is German mosque-sermon subtitling (de-DE); general-domain translation quality is inherited from the base model but not re-validated.
• Trained for ar → de only.
• Short-chunk inputs (live ASR segments) are the design case; long documents untested.
• In MinbarAI, Quran verses and fixed liturgical texts are served from canonical translations before reaching this model — the model's job is free speech.

License

Gemma Terms of Use (inherited from the base model). Quran translation data from Tanzil via fawazahmed0/quran-api.