small-100-Singlish-Sinhala-CodeMix2-ONNX-INT8

small-100 Singlish → Sinhala (ONNX INT8)

Fine-tuned and quantized small-100 seq2seq model for translating Roman-script Sinhala (Singlish) mixed with English into pure Sinhala Unicode script. Exported to ONNX and quantized to INT8 for fast CPU inference.

---

What it does

Takes informal Singlish text typed by Sri Lankans on social media and converts it to Sinhala script. English words are translated to their Sinhala meaning — not transliterated phonetically.

---

Training details

• Base model: alirezamsh/small100 (encoder-decoder seq2seq)
• Fine-tuning: LoRA (r=64, alpha=128) on combined Singlish-Sinhala dataset
• Training data:
  • Real social media code-mix: ~26,700 rows × 4 upsample
  • Phonetic transliteration replay: 20,000 rows
  • Adhoc pairs replay: 5,000 rows
  • Total: ~211,000 training samples
• Evaluation: Golden test set of 2,549 real Singlish sentences
• Best CER: ~13.8% on golden test set

Export details

• Exported from merged LoRA fp32 model to ONNX
• INT8 dynamic quantization using AVX512 VNNI kernels via optimum
• Each ONNX file quantized separately: encoder, decoder, decoder_merged
• Greedy decoding (num_beams=1) recommended for CPU inference

---

Installation

pip install optimum[onnxruntime] transformers sentencepiece

---

Usage

Basic inference

from optimum.onnxruntime import ORTModelForSeq2SeqLM
from transformers import AutoTokenizer

model_id = "savinugunarathna/small-100-Singlish-Sinhala-CodeMix2-ONNX-INT8"

model    = ORTModelForSeq2SeqLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.src_lang = "en"
tgt_lang_id = tokenizer.lang_code_to_id["si"]

def translate(text):
    inputs = tokenizer(
        text,
        return_tensors="pt",
        truncation=True,
        max_length=128,
    )
    outputs = model.generate(
        **inputs,
        max_new_tokens=64,
        num_beams=1,           # greedy — fastest on CPU
        forced_bos_token_id=tgt_lang_id,
    )
    return tokenizer.decode(outputs[0], skip_special_tokens=True).strip()

print(translate("mama hungry, kanna yamu"))
# → මම බඩගිනියි, කෑම කමු

Batch inference

texts = [
    "mama hungry, kanna yamu",
    "bro API eka call karanna puluwanda",
    "today weather eka honda ne",
]

inputs = tokenizer(
    texts,
    return_tensors="pt",
    padding=True,
    truncation=True,
    max_length=128,
)
outputs = model.generate(
    **inputs,
    max_new_tokens=64,
    num_beams=1,
    forced_bos_token_id=tgt_lang_id,
)
for out in outputs:
    print(tokenizer.decode(out, skip_special_tokens=True).strip())

FastAPI server example

from fastapi import FastAPI
from pydantic import BaseModel
from optimum.onnxruntime import ORTModelForSeq2SeqLM
from transformers import AutoTokenizer

app       = FastAPI()
model_id  = "savinugunarathna/small-100-Singlish-Sinhala-CodeMix2-ONNX-INT8"
model     = ORTModelForSeq2SeqLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.src_lang  = "en"
tgt_lang_id = tokenizer.lang_code_to_id["si"]

class Request(BaseModel):
    text: str

@app.post("/translate")
def translate(req: Request):
    inputs = tokenizer(req.text, return_tensors="pt", truncation=True, max_length=128)
    outputs = model.generate(
        **inputs,
        max_new_tokens=128,
        num_beams=3,
        forced_bos_token_id=tgt_lang_id,
    )
    result = tokenizer.decode(outputs[0], skip_special_tokens=True).strip()
    return {"input": req.text, "output": result}

Run with:

uvicorn app:app --host 0.0.0.0 --port 8000

---

Performance

Benchmarked on CPU (single thread, greedy decoding, max_new_tokens=64):

---

Limitations

• Optimized for short conversational sentences (5-20 words)
• Very long sentences (50+ words) may see quality degradation
• Rare English words without common Sinhala equivalents may be transliterated rather than translated
• Conjunct consonants (e.g. ශ්‍රී, ක්‍රිකට්) may occasionally render without ZWJ joiner depending on font rendering

---

Related models

• small-100-Singlish-Sinhala-CodeMix2 — full fp32 PyTorch version of this model
• Small100-Singlish-Sinhala-Merged — base merged model before code-mix fine-tuning