QuranKarim-SpeechToText-onnxModel

QuranKarim-SpeechToText-fastconformer (Mobile-Ready Mixed ONNX)

Alhamdulillah. This repository provides highly optimized, mobile-ready ONNX exports of the FastConformer Hybrid Large model, fine-tuned specifically for offline Quranic Arabic speech recognition and verse tracking.

By utilizing a state-of-the-art Mixed Quantization (INT4 + INT8) pipeline, the model has been compressed from its original 458 MB down to just 87 MB. It successfully retains a 98.1% Sequence Accuracy for Quranic verse recognition, making it the perfect lightweight artifact for edge deployment on mobile devices (Android/iOS) and embedded systems without sacrificing Tajweed and Tashkeel precision.

Key Features for Edge Deployment

• Mixed Quantization (87 MB): Employs a two-pass quantization strategy. Massive Attention and Feed-Forward MatMul weights are squashed to 4-bit (INT4), while the highly sensitive Convolution layers remain at 8-bit (INT8). This avoids the accuracy collapse typical of pure INT4 models.
• Offline CTC Decoding: This ONNX export is configured strictly for offline (non-streaming) transcription. It processes the full audio context at once to guarantee the highest possible accuracy for Quranic verses.
• Sherpa-ONNX Ready: The .onnx file comes pre-injected with all required Sherpa-ONNX metadata (feature dims, subsampling factors, sample rates, etc.).
• Tokens Included: A pre-formatted tokens.txt is included in the repository for immediate CTC decoding.

---

Model Details

---

Usage (Sherpa-ONNX)

This model is designed to be dropped directly into a Sherpa-ONNX offline pipeline. Because the metadata is already injected, you do not need to pass complex configuration flags.

C++ / Python / Flutter Integration

When initializing the Sherpa-ONNX offline recognizer, simply point it to the mixed quantized model alongside the tokens file:

import sherpa_onnx

recognizer = sherpa_onnx.OfflineRecognizer.from_nemo_ctc(
    model="qurankarim-fastconformer-mixed.onnx", # 87 MB Mixed Model
    tokens="tokens.txt",
    num_threads=2,
)

# Pass your 16kHz mono audio array
stream = recognizer.create_stream()
stream.accept_waveform(16000, audio_samples)
recognizer.decode_stream(stream)

print(stream.result.text)
Training Background & Progressive UnfreezingThe base model was fine-tuned using a 3-phase progressive unfreezing strategy on a single NVIDIA RTX 4070 Ti (12 GB). This prevented catastrophic forgetting of the base model's Arabic speech representations while allowing it to adapt strictly to Quranic phonetics and Tajweed rules.PhaseLayers UnfrozenStepsLRVal WERPhase 1Top 3 encoder + decoder20005e-50.0038Phase 2Upper half (layers 9–17) + decoder30001e-40.0018Phase 3All layers25005e-50.0014Training Data: tarteel-ai/everyayah — a diverse multi-reciter dataset of complete Quranic recitations at multiple audio qualities, covering all 114 surahs across dozens of reciters.Qualitative ExamplesThe following are exact reference vs. predicted outputs from the validation set. The model transcribed these word-for-word correctly, including full diacritization (Tashkeel):ReferencePredictedوَهُوَ الَّذِي جَعَلَ لَكُمُ اللَّيْلَ لِبَاسًا وَالنَّوْمَ سُبَاتًا وَجَعَلَ النَّهَارَ نُشُورًا✅ Perfectالزَّانِي لَا يَنْكِحُ إِلَّا زَانِيَةً أَوْ مُشْرِكَةً وَالزَّانِيَةُ لَا يَنْكِحُهَا إِلَّا زَانٍ أَوْ مُشْرِكٌ وَحُرِّمَ ذَلِكَ عَلَى الْمُؤْمِنِينَ✅ Perfectإِلَّا مَنْ تَابَ وَآمَنَ وَعَمِلَ عَمَلًا صَالِحًا فَأُولَئِكَ يُبَدِّلُ اللَّهُ سَيِّئَاتِهِمْ حَسَنَاتٍ وَكَانَ اللَّهُ غَفُورًا رَحِيمًا✅ Perfectإِذْ قَالَ لِأَبِيهِ وَقَوْمِهِ مَا تَعْبُدُونَ✅ Perfectيَوْمَ لَا يَنْفَعُ مَالٌ وَلَا بَنُونَ✅ Perfectإِذْ قَالَ لَهُمْ أَخُوهُمْ هُودٌ أَلَا تَتَّقُونَ✅ Perfectأَتَبْنُونَ بِكُلِّ رِيعٍ آيَةً تَعْبَثُونَ✅ PerfectThese span multiple surahs and include some of the most phonetically demanding Ayahs in the Quran—featuring long compound sentences, rare vocabulary (نُشُورًا، سُبَاتًا), emphatic consonants, and precise Tashkeel on every word.Intended Use & LimitationsIntended Use:Offline, on-device mobile applications (Android/iOS/Flutter)Quranic recitation transcription and verse trackingTajweed learning and pronunciation verification appsAyah identification from audioLimitations:Not for real-time streaming: This ONNX export processes the entire audio chunk at once. It is not configured for frame-by-frame streaming.Optimised specifically for Quranic Arabic — performance on Modern Standard Arabic or dialectal Arabic will be significantly lower.Best results are achieved on clean, single-speaker recitation audio at 16kHz. Heavy background noise may induce character stuttering.CitationIf you use this model, please cite the base model and dataset:Code snippet@misc{fastconformer-quran-ar,
  author    = {Mohammed},
  title     = {FastConformer Quran Arabic ASR},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {[https://huggingface.co/mohammed/fastconformer-quran-ar](https://huggingface.co/mohammed/fastconformer-quran-ar)}
}

@misc{everyayah,
  author    = {Tarteel AI},
  title     = {EveryAyah: A Quranic Recitation Dataset},
  publisher = {Hugging Face},
  url       = {[https://huggingface.co/datasets/tarteel-ai/everyayah](https://huggingface.co/datasets/tarteel-ai/everyayah)}
}