Byt5_OOV_finetuned

Byt5_OOV_finetuned

A ByT5-small model fine-tuned to predict morpheme boundaries in Nepali words, with a specific focus on generalizing to roots never seen during training (out-of-vocabulary / OOV generalization).

Given a raw Nepali surface word, the model outputs the word split into root and affix, separated by +:

Input:  अँकाइनु
Output: अँका+इनु

Model Details

Model Description

Nepali morphology means a single dictionary root can surface in many inflected/derived forms. This model is trained to segment a word into its root and affix components at the byte level, which sidesteps the risk of a subword tokenizer fragmenting Devanagari's multi-byte, combining-mark-heavy sequences inconsistently.

• Developed by: w4ashabii
• Model type: Encoder-decoder (T5-family, byte-level), fine-tuned
• Language: Nepali (ne)
• License: MIT
• Finetuned from model: google/byt5-small

Model Sources

• Training data: w4ashabii/root_affix_dictionary

Uses

Direct Use

Segmenting a Nepali surface word into root + affix, e.g. as a preprocessing step for morphological analysis, lemmatization, or downstream NLP pipelines that benefit from root-level normalization.

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("w4ashabii/Byt5_OOV_finetuned")
model = AutoModelForSeq2SeqLM.from_pretrained("w4ashabii/Byt5_OOV_finetuned")

word = "अँकाइनु"
input_ids = tokenizer(word, return_tensors="pt").input_ids
output_ids = model.generate(input_ids, max_length=40)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
# अँका+इनु

Out-of-Scope Use

Not intended for full morphological analysis beyond root/affix segmentation (e.g. no POS tagging, no gloss/meaning output), and not evaluated on languages other than Nepali.

Bias, Risks, and Limitations

• Training data includes both dictionary-verified segmentations and heuristically-inferred ones (see the dataset card); heuristic-derived examples may carry systematic rule-engine errors that the model can inherit.
• Evaluated on a root-disjoint OOV test set drawn from the same dictionary source (10th edition); performance on genuinely novel coinages, loanwords, or dialectal forms outside that source is untested.
• The model slightly under-splits relative to over-splitting (see boundary precision vs. recall below) — it more often misses a boundary than invents a spurious one.

Training Details

Training Data

Fine-tuned on w4ashabii/root_affix_dictionary, split root-disjoint (not just word-disjoint) via grouped splitting on the Root column, so no root appears in more than one split — this is what makes the test set a genuine test of generalization to unseen roots rather than memorization.

• Train: 42,647 examples
• Validation: 5,212 examples
• Test: 5,260 examples

Training Procedure

• Base model: google/byt5-small
• Precision: fp32 (ByT5/T5 models are numerically unstable in fp16; bf16 was unavailable on the training GPU)
• Epochs: trained for 10, best checkpoint selected by validation exact match — peaked at epoch 4 and mildly overfit afterward
• Batch size: 128 train / 64 eval
• Learning rate: 3e-4
• Selection: best checkpoint by validation exact-match accuracy

Hardware

Single NVIDIA T4 GPU (Google Colab)

Evaluation

Evaluated on the root-disjoint OOV test split (5,260 words never seen — by root — during training).

Metrics

Two metrics, matching what matters for partial credit vs. strict correctness:

• Exact Match — predicted segmented string equals gold exactly
• Boundary F1 — treats each + as a boundary at a character offset in the reconstructed word and scores predicted vs. gold boundary positions with precision/recall/F1, so a model that gets 3 of 4 boundaries right on a word scores better than one that gets 0

Results

The gap between boundary precision and recall indicates the model is somewhat more conservative than aggressive — it's more likely to miss a true boundary on an unseen root than to hallucinate one that isn't there.

Citation

BibTeX:

@misc{byt5_oov_finetuned,
  author = {w4ashabii},
  title = {Byt5\_OOV\_finetuned: A ByT5 model for Nepali morpheme boundary prediction},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/w4ashabii/Byt5_OOV_finetuned}}
}