linguaforge-gemma4-204lang-lora

LinguaForge — Gemma 4 E4B LoRA across 204 languages

A single ~170 MB LoRA adapter that shifts Google DeepMind's google/gemma-4-E4B-it toward every language in FLORES-200 (Meta NLLB Team, No Language Left Behind, Nature 2024) plus Cherokee depth from the ChrEn corpus (Zhang, Frey & Bansal, EMNLP 2020). Cherokee is not in FLORES-200.

Trained as part of the LinguaForge / 古韵 GuYun submission to the Gemma 4 Hackathon (AI for Good — endangered language preservation).

Training summary (Kaggle T4, ~5 h 9 min)

Held-out evaluation (FLORES-200 devtest + ChrEn seed=99)

Numbers from Kaggle kernel dongwei666/linguaforge-eval, 50 unseen sentences per language, greedy decoding, sacrebleu corpus-level metrics.

Honest read: the LoRA's biggest wins are on languages whose scripts the base model could barely write (Cherokee chrF 3.4×, Tibetan chrF +7.91). Welsh shows the largest BLEU jump (+2.23) — the adapter strips a **Welsh Translation:** boilerplate prefix from the base model. Yoruba regressed into a repetition loop; reported transparently. With more samples per language or per-community LoRAs, that regression should resolve.

Usage

from unsloth import FastLanguageModel
from unsloth.chat_templates import get_chat_template
import torch

model, tok = FastLanguageModel.from_pretrained(
    model_name="zcgf111/linguaforge-gemma4-204lang-lora",
    max_seq_length=2048,
    load_in_4bit=True,
)
tok = get_chat_template(tok, chat_template="gemma")
FastLanguageModel.for_inference(model)

msgs = [
    {"role": "system", "content": "You are LinguaForge, a multilingual tutor for endangered and low-resource languages."},
    {"role": "user",   "content": "Translate this English sentence into Maori (Polynesian, Pacific):\n\nHello, my name is Sarah."},
]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inputs = tok.tokenizer(text, return_tensors="pt").to(model.device)
with torch.inference_mode():
    out = model.generate(**inputs, max_new_tokens=128, do_sample=False,
                         pad_token_id=tok.tokenizer.eos_token_id)
print(tok.tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Citations

@article{nllb2024,
  title={Scaling neural machine translation to 200 languages},
  author={{NLLB Team} and Costa-juss{\`a}, Marta R. and others},
  journal={Nature},
  year={2024},
  doi={10.1038/s41586-024-07335-x}
}
@inproceedings{zhang-etal-2020-chren,
  title={{ChrEn}: {Cherokee-English} Machine Translation for Endangered Language Revitalization},
  author={Zhang, Shiyue and Frey, Benjamin and Bansal, Mohit},
  booktitle={EMNLP},
  year={2020}
}

License

CC-BY-SA 4.0, matching the FLORES-200 license. ChrEn is released under CC-BY-SA 4.0 by its authors.