pantogloss-500-en-v6

Pantogloss 500-to-English successor v6

This repository contains the released full-precision v6 fine-tune. It is separate from the historical parent and Compact models; the Pantogloss package default remains Compact. The weights are hosted here, not in the Python wheel.

Usage

from huggingface_hub import snapshot_download
from pantogloss import Translator

model_dir = snapshot_download("chrismattmann/pantogloss-500-en-v6")
translator = Translator.from_directory(model_dir, device="auto")
print(translator.translate("Comment allez-vous ?"))

What changed

V6 starts from the original RTG many-to-English parent, runs 1,000 bounded PyTorch fine-tuning updates on 2,000 approved examples (250 unique pairs each for Amharic, Hausa, Igbo, Khmer, Lao, Burmese, Yoruba, and Zulu), then maps the learned weights back into Pantogloss TensorFlow/Keras. It uses the same architecture and tokenizers. These are existing source languages being improved, not eight newly added languages; English remains the output language. The PyTorch-to-Keras logit check passed with maximum absolute error 1.00e-5. This released model is full-precision FP32; no compact v6 artifact has been created, and the released Compact model and package default are unchanged.

“500-to-English” describes model-family training provenance, not a claim of uniform quality across 500 languages. The evaluation below covers 50.

Evaluation

On the pinned 8,250-sentence FLORES+ 4.6 devtest comparison against the immutable full-precision parent, matched greedy decoding gave +0.55 aggregate chrF, +0.72 aggregate BLEU, and zero failed or empty translations. All eight target languages improved. Their chrF deltas were Amharic +0.38, Hausa +1.51, Igbo +0.94, Khmer +2.48, Lao +0.50, Burmese +0.73, Yoruba +1.33, and Zulu +0.53; the target median was +0.83.

Pinned COMET rose from 0.83065 to 0.83478 (+0.00413). The COMET model does not declare coverage for Igbo, Yoruba, or Zulu; their per-language COMET scores are not used as quality claims. Korean (−0.34), Russian (−0.13), Vietnamese (−0.08), and Romanian (−0.03) had small chrF declines. These are real observed regressions.

On this evaluation split, the mean sentence-chrF paired-bootstrap difference was +0.55 (95% interval +0.44 to +0.66). This interval is not for corpus chrF or across-training-seed variability. Several individual target intervals include zero; aggregate improvement should not be read as certainty for every sentence or domain.

The exact seven-file staging artifact passed local Linux CPU/CUDA and Apple M3 CPU/Metal functional checks. On a 12-example authored weak-language fixture, M3 greedy and beam-4 outputs matched CUDA exactly with zero failures. This fixture is a decoder smoke test, not a representative quality benchmark; the 8,250-example quality comparison above was run on CUDA.

Known limitations

• The model is for research and general-purpose text translation, not safety-critical, medical, legal, or unreviewed high-stakes use.
• Khmer confirmation examples show phrase repetition in both models; one v6 greedy output repeats markedly longer. Other fluent-looking examples differ in specific details and have not had native-speaker source review. Automated source-copy and output-length checks are diagnostic only, not proof that hallucinations are absent.
• Quality varies by language, domain, script, and sentence. This model does not detect source language or provide calibrated confidence.
• The 50-language evaluation does not support uniform-quality claims about every language in the parent model family.

Provenance and redistribution

The parent is the RTG many-to-English model described by Gowda, Zhang, Mattmann, and May, ACL-IJCNLP 2021 System Demonstrations. Pantogloss's new model artifact is released under Apache-2.0. Training-source attribution for the exact v6 mix is in the accompanying V6-ATTRIBUTION.md. Detailed filtering/transformation records are maintained privately in a pinned data ledger; the public attribution notice is self-contained. Training corpus text and benchmark text are not included in this artifact.

Release record

This model revision passed artifact-integrity and CPU/CUDA/Metal functional checks. The model is not the Pantogloss package default.