swahili-kenlm

🗣️ Swahili KenLM Language Model

This repository contains a KenLM n-gram language model trained on a large Swahili text corpus (~11.3M sentences) from Benjamin-png/swahili-normalized-corpus.
The model is designed for ASR decoding, text prediction, and other NLP tasks involving Swahili.

---

📊 Model Details

• Language: Swahili (sw)
• Model Type: KenLM 3-gram model
• Training Data: 11.3 million normalized Swahili sentences
• Training Tool: KenLM
• Smoothing: Modified Kneser–Ney
• Pruning: Applied for higher-order n-grams to reduce memory usage

---

📂 Files

• swahili_3gram_lm.arpa – ARPA format language model
• swahili_3gram_lm.bin – Binary format for fast loading in decoders (recommended)

---

🚀 Usage

Python (with kenlm library)

!pip install pyctcdecode==0.5.0
!pip install https://github.com/kpu/kenlm/archive/master.zip

import kenlm
KENLM_REPO = "DarthGlennium/swahili-kenlm"
KENLM_FILENAME = "swahili_3gram_lm.bin"

kenlm_path = hf_hub_download(repo_id=KENLM_REPO, filename=KENLM_FILENAME)