burmese-pos-xlmr

Myanmar POS tagger (XLM-RoBERTa)

Fine-tuned XLM-RoBERTa for Myanmar (Burmese) part-of-speech tagging on myPOS, plus high-confidence pseudo labels from unlabeled book text.

Install (weights — this repo)

pip install torch transformers burmesenlp

---

Usage (transformers)

Word-segment with burmesenlp first, then tag. Myanmar script has no spaces between words; the model labels tokens, it does not choose boundaries.

Use Unicode text. Do not run zg2uni unless the source is known Zawgyi (false conversion breaks segmentation).

import torch
from burmesenlp import word_tokenize
from transformers import AutoModelForTokenClassification, AutoTokenizer

name = "aungthuhein-dev/burmese-pos-xlmr"
tokenizer = AutoTokenizer.from_pretrained(name)
model = AutoModelForTokenClassification.from_pretrained(name)
model.eval()

text = "ဒေါ်အောင်ဆန်းစုကြည်သည်နိုင်ငံတော်အတိုင်ပင်ခံဖြစ်သည်။"
words = [w for w in word_tokenize(text) if w.strip()]

inputs = tokenizer(
    words,
    is_split_into_words=True,
    return_tensors="pt",
    truncation=True,
    max_length=128,
)
with torch.no_grad():
    pred_ids = model(**inputs).logits.argmax(dim=-1)[0].tolist()

id2label = {int(k): v for k, v in model.config.id2label.items()}
tags, prev = [], None
for i, wid in enumerate(inputs.word_ids()):
    if wid is None or wid == prev:
        continue
    tags.append(id2label[pred_ids[i]])
    prev = wid

print(list(zip(words, tags)))

Example output:

[('ဒေါ်အောင်ဆန်းစုကြည်', 'n'), ('သည်', 'ppm'), ('နိုင်ငံတော်', 'n'),
 ('အတိုင်ပင်ခံ', 'n'), ('ဖြစ်', 'v'), ('သည်', 'ppm'), ('။', 'punc')]

Tag the first subword of each word only. Do not pass a list of words into pipeline(...) (that treats each word as a separate sentence).

Pipeline

Unicode sentence
  → NFC + strip ZWJ/ZWNJ
  → burmesenlp.word_tokenize
  → XLM-RoBERTa (first subword = word tag)
  → [(word, tag), ...]

---

Tagset (myPOS, 15 labels)

---

Training

1. Train xlm-roberta-base on myPOS gold (43,196 sentences) → baseline.
2. Tag unlabeled book sentences; keep examples where every word-level softmax ≥ 0.95 → 205,721 pseudo-labeled sentences.
3. Fine-tune on gold myPOS + pseudo labels → this checkpoint.

---

Evaluation

Word-level POS on a 10% myPOS holdout (train_test_split, seed=42, 4,320 sentences, 52,247 tokens). First subword of each word carries the tag; 29 sentences skipped (length / alignment at 128 subwords).

Pseudo-labeling does not improve in-domain myPOS F1 (the gold-only baseline is already saturated). Extra data targets book-style vocabulary at inference, not a higher official test score.

---

Limitations

• In-domain myPOS F1 is tied with the gold-only baseline.
• Social, chat, and code-mixed text are not the training domain.
• Inference uses burmesenlp.word_tokenize; pseudo-label training data used a different segmenter.
• Rare tags (abb, int, sb) have very small support on the holdout.

---

Citation

myPOS: Ye Kyaw Thu et al. https://github.com/ye-kyaw-thu/myPOS
Model: https://huggingface.co/aungthuhein-dev/burmese-pos-xlmr

License

Check this model card and the myPOS corpus terms before redistribution.