auto-spello-fa

Auto Spello FA 🚀

The Ultimate Context-Aware Phonetic Spell-Checker for Persian ASR Systems

auto_spello_fa is a high-performance, purely Rust-based NLP engine wrapped in Python. It is specifically architected to correct phonetic hallucinations and speech recognition errors produced by ASR (Automatic Speech Recognition) models like OpenAI Whisper, operating at lightning speeds (< 0.05s inference time).

🧠 The ASR Challenge

Traditional spell-checkers rely on Keyboard Levenshtein distance (e.g., correcting "تایپ" to "تابپ"). However, ASR systems like Whisper do not make keyboard typos; they make phonetic errors based on audio frequencies. For instance, Whisper might transcribe "آسمان" (Sky) as "آفمان", or "قصد" (Intention) as "غصد".

Standard dictionary lookups fail here. To solve this, we need a system that understands the phonetic nature of the Persian language and the contextual probability of words.

⚙️ Methodology & Architecture

This engine relies on a 600MB highly compressed, pre-compiled binary model, extracted from a massive 10GB Persian corpus (120+ million sentences).

1. Phonetic Hash-Mapping (O(1) Memory Safe)

Instead of generating an exponential number of phonetic permutations (which leads to combinatorial explosion and RAM overrun), this engine collapses phonetically similar Persian characters into a unified root hash.

• Sibilants: [س, ص, ث, ش, ف] -> س
• Z-sounds: [ز, ذ, ض, ظ, ژ, ج] -> ز
• Gutturals: [ق, غ, خ] -> ق

If Whisper outputs "آفمان", the engine converts it to the phonetic hash احمام. It then performs an $O(1)$ lookup to find all valid dictionary words sharing this exact phonetic signature (e.g., "آسمان").

2. Context Intersection (Bigram N-Grams)

Phonetic matching alone is insufficient. If the engine finds multiple valid candidates (e.g., "خواننده" and "راننده" for the hallucination "خاننده"), it utilizes a massive Pre-Calculated Bigram Index. The algorithm calculates the probability of each candidate by analyzing the Intersection of Shared Words: It looks at the word immediately preceding and immediately succeeding the typo. By cross-referencing these neighbors with the Bigram Index of the correct candidates, the engine confidently disambiguates the intended word.

$$Score = (\log_{10}(Freq_{unigram}) + 1) \times \sum (Freq_{bigram_context} \times 100)$$

3. Rust-Powered Speed

By bypassing Python's GIL and memory management, the Rust core maps the 600MB binary dictionary directly into memory, enabling real-time subtitle processing without bottlenecking GUI applications or video rendering pipelines.

📦 Installation

1. Install the Python Package:

pip install auto_spello_fa

2. Download the Pre-compiled Phonetic Model:

Download the spello_model_v1.bin (approx. 600MB) from our Hugging Face repository:

👉 Download spello_model_v1.bin

🚀 Usage

import auto_spello

# 1. Initialize the engine with the downloaded model
# Loads in < 0.1 seconds thanks to Rust's Bincode serialization
spell_checker = auto_spello.AutoSpello("path/to/spello_model_v1.bin")

# 2. Feed the raw ASR output
whisper_output = "این صداهای بم تر صدای خاننده است"
corrected_text = spell_checker.correct_sentence(whisper_output)

print(f"Whisper Output : {whisper_output}")
print(f"Corrected Text : {corrected_text}")
# Output: این صداهای بم تر صدای خواننده است

🛠 Building from Source (Rust Developers)

If you wish to re-compile the 10GB corpus or build the package from source:

# Clone the repository
git clone https://github.com/ama1372/auto_spello_fa.git
cd auto_spello_fa

# Build the python wheel
pip install maturin
maturin build --release --out dist

📜 License

This project is open-source and available under the MIT License.