AmanLeb

AmanLeb

AmanLeb is a multilingual SMS classification model developed for scam and smishing detection in Lebanon.

The model is fine-tuned from:

distilbert/distilbert-base-multilingual-cased

It predicts three classes:

• ham
• spam
• smishing

Final Test Performance

• Accuracy: 97.3%
• Macro F1: 0.905
• Suspicious messages predicted as HAM: 2
• Smishing messages predicted as HAM: 0 on the held-out test set

Intended Use

This model is part of the AmanLeb research prototype for detecting suspicious SMS messages in multilingual settings.

The full project, including notebooks, Streamlit application, RAG pipeline, evaluation results, and documentation, is available in the AmanLeb GitHub repository.

Limitations

This model was trained on a relatively small SMS phishing dataset and should not be treated as a production anti-fraud system.

Performance may degrade on:

• unseen scam patterns
• dialectal Arabic
• Arabizi variations
• new organizations or brands
• adversarial messages
• distribution shifts

The reported zero smishing-to-HAM errors applies only to the held-out test set and is not a guarantee for real-world messages.

Base Model

distilbert/distilbert-base-multilingual-cased