voho-saudi-speak-0.6b-ONNX

Voho Saudi Speak 0.6B — ONNX

ONNX builds of Voho Saudi Speak 0.6B, for ONNX Runtime and Optimum.

Turns formal Arabic into Arabic the way Saudis actually say it, in Najdi, Hijazi or Khaleeji. Give it a formal sentence and a dialect, and it returns what a person from Riyadh, Jeddah or the Eastern Province would say on a phone call.

Files

Exported with Optimum, with KV cache. Both were checked against the original model: the full-precision build reproduces it exactly, and int8 matched on two of the three test sentences, differing by one synonym on the third.

Usage

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForCausalLM

repo = "VohoAI/voho-saudi-speak-0.6b-ONNX"
tok = AutoTokenizer.from_pretrained(repo)
model = ORTModelForCausalLM.from_pretrained(repo)          # or file_name="model_int8.onnx"
model.embed_size_per_head = 128                             # see the note below

DIALECT = {"najdi": "النجدية", "hijazi": "الحجازية", "khaleeji": "الخليجية الشرقية"}

def saudi(text, dialect="najdi"):
    prompt = f"أعد صياغة هذه الجملة باللهجة السعودية {DIALECT[dialect]} كما يقولها شخص في مكالمة، بدون أي شرح:\n{text}"
    x = tok.apply_chat_template([{"role": "user", "content": prompt}], add_generation_prompt=True,
                                enable_thinking=False, return_tensors="pt", return_dict=True)
    out = model.generate(**x, max_new_tokens=96, do_sample=False)
    return tok.decode(out[0, x["input_ids"].shape[1]:], skip_special_tokens=True).strip()

print(saudi("أين أنت الآن؟ أريد أن أحجز موعداً غداً."))
# وينك الحين أبغى أحجز موعد بكرة

The one extra line

model.embed_size_per_head = 128 is not optional. Optimum works the attention size out as hidden_size / num_attention_heads, which gives 64 here, while this architecture sets head_dim to 128 explicitly. Without the line, generation stops with:

INVALID_ARGUMENT : Got invalid dimensions for input: past_key_values.8.value
index: 3 Got: 64 Expected: 128

The prompt

Use greedy decoding (do_sample=False) and enable_thinking=False.

Examples

Results, training details and limitations are on the main model card.

Licence

Non-commercial, CC BY-NC-SA 4.0, inherited from the SADA training data. For production Saudi Arabic voice, use the Voho API.

Please cite SADA: Saudi Audio Dataset for Arabic (SADA), Saudi Data and AI Authority (SDAIA), 2022.