vn-diacritic-vit5-base

nrl-ai/vn-diacritic-vit5-base — Vietnamese diacritic restoration (ViT5 fine-tune)

Restores diacritics on Vietnamese text written without them (Toi yeu Viet Nam → Tôi yêu Việt Nam). Fine-tuned from VietAI/vit5-base on a register-balanced mix of Vietnamese Wikipedia (CC-BY-SA-4.0) and Vietnamese news (tmnam20/Vietnamese-News-dedup, CC-BY-4.0, NFC-normalized).

Adoption gate: ✅ passed.

Quick start

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tok = AutoTokenizer.from_pretrained("nrl-ai/vn-diacritic-vit5-base")
model = AutoModelForSeq2SeqLM.from_pretrained("nrl-ai/vn-diacritic-vit5-base").eval()

text = "Toi yeu Viet Nam"
out = model.generate(**tok(text, return_tensors="pt"), max_length=256)
print(tok.decode(out[0], skip_special_tokens=True))
# Tôi yêu Việt Nam

For the full pipeline (with rule-based + LLM fallbacks), use the nom-vn Python package:

from nom.text.diacritic_models import HFDiacriticModel
restorer = HFDiacriticModel(model_id="nrl-ai/vn-diacritic-vit5-base")
restorer("Toi yeu Viet Nam")  # 'Tôi yêu Việt Nam'

Evaluation — 4-register matrix

Measured against Toshiiiii1/Vietnamese_diacritics_restoration_5th (public SOTA at the time of training). Word accuracy after Unicode NFC

• punctuation normalization on both sides.

Each eval corpus is open-license and reproducible from the nom-vn repo:

• business_55 — benchmarks/data/diacritic_eval_v0.txt (CC0)
• literary_udvtb — benchmarks/data/ud_vi_vtb/test.conllu (CC-BY-SA-4.0)
• conversational_300 — benchmarks/data/tatoeba_vi/diacritic_eval_300.txt (CC-BY 2.0 FR)
• formal_udhr — benchmarks/data/udhr_vi/diacritic_eval_udhr.txt (public domain)

How we compare

Where this model sits in the public Vietnamese diacritic-restoration landscape — same 4-register grid for every measured row. Bold = best in column. Cells marked "—" weren't run on that register.

Training

• Base: VietAI/vit5-base (MIT license)
• Corpus: 594,945 (input, target) pairs from a register-balanced mix of Vietnamese Wikipedia (CC-BY-SA-4.0) and Vietnamese news (tmnam20/Vietnamese-News-dedup, CC-BY-4.0, NFC-normalized). Eval-leak guarded against the held-out nrl-ai/vn-diacritic-eval slices.
• Validation: 5,000 held-out pairs from the same training mix.
• Epochs: 5
• Effective batch size: 32 (32 per device, grad-accum 1)
• Learning rate: 0.0005 with cosine schedule, 500 warmup steps
• Precision: bf16
• Sequence length: input 256, target 256
• Early stopping: patience=0 on eval_loss
• Training time: 231.8 min on a single NVIDIA RTX 3090 24 GB
• Seed: 42

Intended use

• Recommended: Restoring diacritics in modern Vietnamese text where the input is predominantly diacritic-stripped (e.g. user typed without IME, OCR with no diacritic-aware font, or imported foreign-keyboard data).
• Not recommended: Spelling correction (the model does not change letters, only adds tone/vowel marks), text generation, classification, or any task the input distribution doesn't match (heavy emoji / mixed-script text).

Limitations

• Register skew. Encyclopedic Wikipedia training tilts the model toward formal/literary register. Conversational / dialect text may be slightly worse.
• Proper noun ambiguity. Multiple plausible diacritisations exist for the same ASCII form (Hung → Hùng / Hưng / Hứng). The model picks the most-likely-in-training, which may not match a specific person's name.
• Long sentences truncate at 256 sub-word tokens. Split paragraphs at sentence boundaries before calling.

License & attribution

This fine-tune is released under Apache 2.0.

Base model: VietAI/vit5-base by VietAI, MIT license. Cite both this fine-tune and the base model:

@misc{vit5_2022,
  title={ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generation},
  author={Long Phan and Hieu Tran and Hieu Nguyen and Trieu H. Trinh},
  year={2022},
  note={NAACL-SRW}
}

@misc{nom_vn_diacritic_2026,
  title={Vietnamese Diacritic Restoration — register-balanced ViT5 fine-tune},
  author={Nguyen, Viet-Anh and {Neural Research Lab}},
  year={2026},
  howpublished={\url{https://huggingface.co/nrl-ai/vn-diacritic-vit5-base}}
}

Training corpus license: CC-BY-SA-4.0 (Wikipedia). Output text from this model is therefore best treated as CC-BY-SA-4.0 if you want to be safe about derivative-rights propagation, even though the model weights themselves are Apache 2.0.