Naqta

Naqta - Arabic Punctuation Restoration

Restores punctuation in Modern Standard Arabic. Two components in one repo:

Usage

from transformers import AutoModelForTokenClassification, AutoTokenizer

tok = AutoTokenizer.from_pretrained("MostafaMaroof/Naqta")
model = AutoModelForTokenClassification.from_pretrained("MostafaMaroof/Naqta")

span_tok = AutoTokenizer.from_pretrained("MostafaMaroof/Naqta", subfolder="span-tagger")
span_model = AutoModelForTokenClassification.from_pretrained(
    "MostafaMaroof/Naqta", subfolder="span-tagger")

Use sliding-window inference for long text. The model was fine-tuned on short windows; a single truncated pass leaves everything past roughly 230 words unpunctuated. See app.py in the Space for a reference implementation.

Evaluation

Two evaluations on deliberately different data. The gap between them is the main practical finding: Arabic punctuation is convention-dependent, so in-domain scores overstate performance on text punctuated to an external editorial standard.

1. In-domain test set

Held-out split of the training corpus (books, Wikipedia, diacritised text) — same sources and editorial conventions as training.

2. External guideline benchmark

91 Modern Standard Arabic passages (about 27k tokens) annotated by an independent Arabic language centre in Saudi Arabia, following its own published punctuation guideline. No overlap with the training data.

Per-class F1 (current):

The drop from 0.896 to 0.616 is concentrated in marks whose placement depends on editorial convention rather than syntax. Structurally determined marks (question mark, colon) transfer well; the semicolon and exclamation, whose rules differ between style guides, do not.

Limitations

• Exclamation and semicolon are over-predicted (precision 0.204 and 0.362): the model finds these marks but does not withhold them reliably.
• Brackets and quotation marks are not produced. The span tagger covers them in training but their precision is too low to enable in production.
• Expect lower accuracy on text following a punctuation standard different from the training sources.

Training

XLM-RoBERTa-large fine-tuned on about 1.3M paragraphs (classical and modern books, Wikipedia, news, QA corpora) with weighted focal loss, oversampling of rare marks, and a final adaptation stage on guideline-conforming data.