xlmr-intent-v2

Ferman intent classifier (XLM-R base, v2)

Classifies assistant commands into 34 intents in English, Arabic, Sorani Kurdish and Badini Kurdish. Fine-tuned from xlm-roberta-base. Part of the Ferman project: https://github.com/omarGH99/ferman-ai-assistant

Usage

from transformers import pipeline
clf = pipeline("text-classification", model="OmarSY11/xlmr-intent-v2", top_k=3)
print(clf("set an alarm for six thirty am"))
print(clf("ئەلارمێکی نوێ بۆ شەشی بەیانی"))      # Sorani

Training data

The v2 configuration of OmarSY11/ferman-nlu, train split. English and Arabic derive from MASSIVE; Sorani and Badini are machine-translated (see the dataset card for provenance and limits).

Training

The notebook is in the repo: ml/train_xlmr_intent_v2_runpod.ipynb.

Results

Held-out test split of the v2 data (4,648 rows).

Kurdish test sets are small, so those figures carry wide uncertainty. The weakest intent is general_quirky, a catch-all class (F1 0.62 overall, 0.00 in Sorani). Per-intent results and the confusion matrix are in the repo under ml/results/.

Limitations

• Kurdish training and test data are machine-translated and informally reviewed by one person (native Badini, non-native Sorani).
• Sorani is the weakest language and is the one v2 did not improve over v1.
• 34 intents only. Out-of-domain input is rejected in the app by confidence and margin thresholds, not by the model itself.
• Not evaluated on real user speech or on other Kurdish varieties.

License and citation

Model weights: MIT, like the base model. Training data: CC BY 4.0, derived from MASSIVE (FitzGerald et al., 2022, arXiv:2204.08582), which must be credited. Please also cite https://github.com/omarGH99/ferman-ai-assistant.