punctuator_model

Kyrgyz Punctuation Restoration — XLM-RoBERTa

The first punctuation restoration model for the Kyrgyz language, achieving 94.1% precision and 90.3% F1-score — surpassing benchmarks for other low-resource languages.

📄 Published research: "AI-Based Punctuation Restoration using Transformer Model for Kyrgyz Language" — Uvalieva Z., Muhametjanova G. (SCOPUS-indexed)

---

Highlights

• 🏆 F1-score: 90.3% — outperforms comparable low-resource language models
• 🌍 First-of-its-kind for Kyrgyz (Turkic language family, ~7M speakers)
• ⚡ ONNX format — optimized for fast inference across frameworks
• 🎙️ ASR post-processing — designed to restore punctuation in speech-to-text output

---

Performance

Cross-Lingual Comparison

The model demonstrates strong performance on frequent punctuation marks (periods, commas) with reduced accuracy on rare marks (question marks, exclamation points) due to class imbalance.

---

Model Architecture

---

Training Details

Dataset

A custom-built 200 MB Kyrgyz text corpus, collected over 2 months:

Preprocessing pipeline: PDF → EasyOCR text extraction → manual cleaning → JSON formatting with punctuation labels.

Data Augmentation

Specialized augmentation techniques designed for Kyrgyz agglutinative morphology:

• Back-translation: Kyrgyz → English → Kyrgyz (simulating ASR-like errors)
• Token-level modifications: Random insertions, deletions, swaps
• Morphological transformations: Case form and morpheme modifications preserving grammatical correctness

Hyperparameters

---

How to Use

import onnxruntime as ort
import numpy as np

# Load the ONNX model
session = ort.InferenceSession("model.onnx")

# Prepare input (see config.yaml for tokenizer settings)
# The model predicts punctuation labels for each token:
# O (no punctuation), COMMA, PERIOD, QUESTION, EXCLAMATION

# Example inference
input_text = "бул кыргыз тилиндеги текст"
# Tokenize and run inference (see main.py for full pipeline)

Repository Structure

├── model.onnx           # Trained model in ONNX format (1.11 GB)
├── main.py              # Inference pipeline
├── env.py               # Environment configuration
├── config.yaml          # Hyperparameters and model config
├── requirements.txt     # Python dependencies
└── Files/               # Additional model files

---

Intended Use

---

Limitations

• Rare punctuation marks: Lower accuracy on question marks and exclamation points due to class imbalance in training data
• Formal text bias: Trained primarily on literary/formal text; performance on informal/conversational text (social media, chat) may be lower
• Morpheme boundary errors: Occasional difficulty placing punctuation in complex agglutinative constructions
• Domain specificity: Best performance on prose-style text; specialized domains may require additional fine-tuning

---

Future Directions

• Joint training with related Turkic languages (Kazakh, Uzbek, Turkish) for improved cross-lingual transfer
• Morphology-aware tokenization to replace standard BPE
• Expanded dataset with informal and conversational Kyrgyz text
• Integration with Kyrgyz ASR systems for end-to-end speech processing

---

Citation

@article{uvalieva2024punctuation,
  author    = {Uvalieva, Zarina and Muhametjanova, Gulshat},
  title     = {AI-Based Punctuation Restoration using Transformer Model for Kyrgyz Language},
  year      = {2024},
  institution = {Kyrgyz-Turkish Manas University}
}

---

Author

Zarina Uvalieva — ML Engineer specializing in NLP and Speech Technologies for low-resource languages.

• 🤗 HuggingFace
• 📧 zarina.uvalievaa@gmail.com