Cohere-Jordanian-Dialect

Cohere Jordanian Dialect

Cohere Jordanian Dialect is a full fine-tune of CohereLabs/cohere-transcribe-arabic-07-2026, specialized for Jordanian dialectal Arabic speech-to-text transcription. It is a 2B-parameter Conformer encoder / Transformer decoder ASR model, fine-tuned end-to-end to substantially improve accuracy on Jordanian dialect audio over the base checkpoint.

This repository is self-contained — it includes the fine-tuned weights plus all processor/tokenizer files needed to run it directly, with no dependency on the base repo at load time.

Quick Start

pip install "transformers==5.13.0" torch huggingface_hub soundfile librosa sentencepiece protobuf accelerate

Note: transformers must be pinned to 5.13.0 — later (5.15.0+) and earlier versions are not supported.

You'll also need a Hugging Face access token (read access) and to have accepted the terms on the base model page before downloading, since the fine-tune derives from that gated repo.

from huggingface_hub import login
login()  # paste your HF token (needs read access)

from transformers import AutoProcessor, CohereAsrForConditionalGeneration
from transformers.audio_utils import load_audio

MODEL_ID = "Rlamas/Cohere-Jordanian-Dialect"

processor = AutoProcessor.from_pretrained(MODEL_ID)
model = CohereAsrForConditionalGeneration.from_pretrained(MODEL_ID, device_map="auto")

# Transcribe Jordanian Arabic audio
audio_file = "your_audio.wav"
audio = load_audio(audio_file, sampling_rate=16000)

inputs = processor(audio, sampling_rate=16000, return_tensors="pt", language="ar")
inputs.to(model.device, dtype=model.dtype)

outputs = model.generate(**inputs, max_new_tokens=256)
text = processor.decode(outputs, skip_special_tokens=True)
print(text)

Architecture

The model is composed of four main stages, plus an 8-bit optimizer used during fine-tuning:

1. Audio feature extractor — converts raw audio into normalized mel-spectrogram features.
2. Audio encoder — a Conformer stack that processes mel-spectrogram features through multiple attention layers to extract acoustic patterns, compressing the sequence length for faster, more efficient downstream processing.
3. Prompt injector — specifies language, punctuation/capitalization, and inverse text normalization, controlling output style and task mode.
4. Text decoder — a lightweight Transformer decoder that translates the extracted audio features directly into Arabic text tokens.
5. 8-bit optimizer (training only) — quantizes optimizer states from 32-bit to 8-bit via bitsandbytes, reducing GPU memory needs during fine-tuning.

Training

Learning rate sweep

A sweep was run over [5e-6, 1e-5, 2e-5, 5e-5], three epochs each, ranked by best CER:

Winner: lr=5e-5, best CER 0.0879 / WER 0.2266 at epoch 2.

Final training run

The winning learning rate (5e-5) was used for a longer run — 5 epochs was insufficient for the model to stabilize, so training was extended to 8 epochs.

Hardware

• GPU: NVIDIA L40S ($2.62/hr)
• CPU: 2 cores
• RAM: 16 GB

Hyperparameters

• Learning rate: 5e-5
• Batch size / gradient accumulation: batch size 8, 16 accumulation steps (effective batch size 128)
• Optimizer: adamw_bnb_8bit
• Precision: BFloat16
• Epochs / warmup: 8 epochs, warmup rate 0.05
• Weight decay: 0.01
• Gradient checkpointing: enabled

Training time: 1 hour 28 minutes.

Results

Evaluated on held-out test data using CER/WER from the jiwer library.

Full fine-tuning delivered a clear accuracy gain on Jordanian dialect audio over the baseline.

Inference Speed

Measured on 10 random records using 1 CPU core, 4GB RAM, and an NVIDIA T4 GPU:

• Mean RTF: 0.2156
• Median RTF: 0.1812

This model does not currently support streaming inference.

Production Deployment

As with the base model, Cohere's model card recommends serving via vLLM for production, which exposes an OpenAI-compatible /v1/audio/transcriptions HTTP endpoint — usable with any client built against the OpenAI Whisper transcription API.

The Test Data

The test data comes from a different category/sector than the training data. It contains different speakers and covers different topics, so the results could be better if the model were evaluated on data from the same domain.

Keep this in mind when looking at the limitations

Limitations & Failure Modes

Over-generation & repetition loops. On long clips, or clips with extended pauses/background noise, the decoder can enter token repetition loops — a known failure mode for autoregressive sequence-to-sequence ASR models.

Reference was an audio full of static

Hypothesis: هاد بدك تدق لا أنا بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدق على الخلطة بدك تدقق على الخلطة بدك تدقق

Single-word audio. Short, single-word clips are prone to misrecognition:

Code-switching / full-English audio. The model can mistranslate or transliterate English audio instead of transcribing it directly:

When to Use This Model

Good fit: offline, batch transcription of long-form Jordanian dialect Arabic audio, where high accuracy (7.8% CER / 18.9% WER) on custom domain data is required and low training overhead matters — this model fully fine-tunes in under 1.5 hours on a single L40S GPU using 8-bit optimization.

Poor fit: real-time conversational streaming, ultra-low-latency applications, or use cases needing word-level timestamps. Its non-streaming autoregressive decoder makes production serving slower and more expensive to scale than streaming CTC or FastConformer alternatives (e.g. Nemotron).

Future Work

To further reduce CER/WER, additional training data targeted specifically at the failure modes above (repetition loops on long/noisy audio, single-word utterances, and code-switched/English audio) is recommended.

Citation

This model is a fine-tune of Cohere Transcribe Arabic. Please cite the base model:

@misc{shaun_cassini_2026,
    author       = { Shaun Cassini and Sebastian Vincent and Xiaolu Lu and Julian Mack and Dhruti Joshi and Pierre Richemond },
    title        = { cohere-transcribe-arabic-07-2026 (Revision 0a8193c) },
    year         = 2026,
    url          = { https://huggingface.co/CohereLabs/cohere-transcribe-arabic-07-2026 },
    doi          = { 10.57967/hf/9549 },
    publisher    = { Hugging Face }
}