translategemma-4b-cherokee-en2chr-lora

TranslateGemma 4B — English→Cherokee (Archivist baseline_v1)

A QLoRA adapter (r=512, α=512) on google/translategemma-4b-it for English → Cherokee (ᏣᎳᎩ) translation. Trained on 287K deduplicated parallel SFT records from the Archivist Cherokee–English corpus.

Cherokee is a polysynthetic, severely endangered Iroquoian language with ~2,000 remaining fluent first-language speakers. This work builds on the foundations laid by the ChrEn project (Zhang et al.), whose parallel corpus and earlier neural machine translation experiments were an inspiration; our contribution is applying modern autoregressive transformer LLMs (via parameter-efficient fine-tuning of a multilingual instruction-tuned base) to the English→Cherokee direction.

Evaluation findings

Locked evaluation, 2026-04-17, on a 192-sample internal no-dialect holdout (released with the dataset as eval_en2chr_holdout_v1.jsonl):

A small dev prompt-sweep (32 samples) scored BLEU 82.3 / chrF 87.8. Script purity measures the fraction of Cherokee-syllabary characters in the output — the model reliably writes in syllabary, not romanization.

Honest caveats: the training corpus skews toward biblical and older registers, so modern colloquial Cherokee is the weakest domain; chrF lags BLEU because Cherokee's polysynthetic morphology penalizes character-level mismatches harshly; and 20.8% exact match means most outputs are near-correct rather than verbatim. This is a research test bed, not a production translator.

Inference contract

Feed plain English input only. Do not prepend an instruction prefix (Translate English to Cherokee:) and do not add dialect tags ([kituwah] / [otali]) — the model was trained without them and behaves best exactly as trained.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "google/translategemma-4b-it", torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(base, "CGICAI/translategemma-4b-cherokee-en2chr-lora")
tok = AutoTokenizer.from_pretrained("CGICAI/translategemma-4b-cherokee-en2chr-lora")

text = "The river flows through our land."
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=128)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

The adapter is saved in bfloat16 and includes the full tokenizer files (the training tokenizer is unchanged from the base model; they are bundled for convenience).

Training details

• Base model: google/translategemma-4b-it
• Method: QLoRA via Unsloth + TRL — r=512, α=512, 4-bit NF4 base, target modules q/k/v/o/gate/up/down_proj, sequence length 512
• Data: 287,416 bidirectional SFT records (clean v1), no dialect tags, plain-English prompt format; 13,455 steps on 2× RTX 4090
• Trained: April 12, 2026

A successor adapter trained on the decontaminated v2 corpus finished training in April 2026 but is not published — development paused after a hardware failure took one training GPU offline, and it has not yet been evaluated. It will be released here once evaluation confirms it improves on this baseline.

Why this project exists

Cherokee has on the order of two thousand fluent first-language speakers remaining. We believe language models can be instruments of preservation — that future models should encompass all languages, carrying historical culture forward within the very weights themselves — and that this must happen through open-source, community-driven work.

Language models tend to flatten the distinctiveness of the cultures they absorb, smoothing every voice into one homogenized register. For endangered languages that flattening is an extinction event in disguise. Preservation has to mean preserving the differences — dialects, registers, and all.

We release this adapter as a test bed for continued work on Cherokee and for other Native American and indigenous languages at risk of being lost. If you build on it, do so in the open, and in partnership with the communities whose language this is.

ᏣᎳᎩ ᎦᏬᏂᎯᏍᏗ ᏱᎩ — it is the Cherokee language.

Citation

@misc{archivist_translategemma_cherokee_2026,
  title  = {TranslateGemma 4B English--Cherokee LoRA (Archivist baseline\_v1)},
  author = {CGICAI},
  year   = {2026},
  url    = {https://huggingface.co/CGICAI/translategemma-4b-cherokee-en2chr-lora}
}