kyrgyz-morph-tagger

Kyrgyz Morphological Tagger — released checkpoint

The seed-42 checkpoint of a morphological tagger for Kyrgyz: it predicts UPOS, a Kyrgyz-specific XPOS, the lemma, and the full Universal Dependencies feature bundle for every token. Syntax (HEAD/DEPREL) is not predicted.

This repository holds weights only. The code that runs them is on PyPI:

pip install kyrgyz-morph-tagger

from kyrgyz_morph_tagger import Tagger

tagger = Tagger.load()          # fetches this checkpoint
for sentence in tagger.tag("Мектептерибизден эрте келдик."):
    for token in sentence:
        print(token.form, token.lemma, token.upos, token.feats_string)

Мектептерибизден  Мектептер  NOUN   Case=Abl|Number=Plur|Poss=Yes
эрте              эрте       ADV    _
келдик            кел        VERB   Mood=Ind|Number=Plur|Person=1|Polarity=Pos|Tense=Past|Voice=Act
.                 .          PUNCT  _

Architecture

A faithful PyTorch re-implementation of UDPipe 2: per word, a character-level BiGRU, a trainable word embedding, and frozen contextual embeddings from xlm-roberta-base (mean of the last four layers, mean subword pooling) feed a 2-layer BiLSTM, with four classification heads — UPOS, XPOS, a lemma edit-rule, and a monolithic FEATS label.

The encoder is frozen, so at inference the package also downloads xlm-roberta-base through transformers. No architectural novelty is claimed: this is a strong, reproducible baseline for the companion corpus.

Accuracy

Test set of the Kyrgyz UD corpus (425 sentences), exact match, gold tokenization:

UFeats is exact-set match of the whole feature bundle; a single wrong feature invalidates the token. That is the honest measure of full morphological analysis in an agglutinative language, and it is why UFeats (81) sits well below UPOS (94).

Under the same exact-match metric, three frontier LLMs given the same tagging scheme in the prompt reach UFeats 58–60 zero-shot, against this tagger's 79.6 on the same subsample.

Training data

The Kyrgyz UD treebank — 8,485 sentences (~98k tokens) with manually annotated gold morphology, split 90/5/5. The corpus is CC BY-SA 4.0 and builds on UD_Kyrgyz-KTMU and UD_Kyrgyz-TueCL.

Licence

These weights are released under the PolyForm Noncommercial 1.0.0 licence: free for research, teaching, personal use and noncommercial organisations. Commercial use requires a separate licence from the licensor. This is not an OSI-approved open-source licence.

The training corpus is licensed separately (CC BY-SA 4.0) and is not covered by this licence.

Limitations

• Morphology only — no syntax.
• Trained on news, web and literary text; other domains will be weaker.
• Raw-text input goes through a rule-based tokenizer (token F1 99.3, 95.3 % of sentences exactly right on the test set). If you already have tokens, use tag_pretokenized() to keep the tokenizer out of the loop.
• Sentences longer than 512 XLM-R subwords lose contextual embeddings for the overflow; the package warns when this happens.

Links

• Code and experiments: https://github.com/AlibekovNurtilek/kyrgyz-morph-tagger
• Corpus: https://github.com/AlibekovNurtilek/kyrgyz-ud-treebank