urdu-roman-transliterator-lora

Urdu to Roman Urdu Transliterator (LoRA Fine-tune)

This model transliterates Urdu script text into Roman Urdu (Urdu written using Latin/English characters).

Model Details

Model Description

This is a LoRA (Low-Rank Adaptation) fine-tuned adapter built on top of ogx786/urdu-roman-transliterator-tiny-aya. It was trained to convert Urdu-script sentences into their Roman Urdu equivalents using an instruction-style prompt format.

• Developed by: [hina]
• Model type: Causal Language Model with LoRA adapter
• Language(s): Urdu (input), Roman Urdu (output)
• License: Apache 2.0 (update if different)
• Finetuned from model: ogx786/urdu-roman-transliterator-tiny-aya

Uses

Direct Use

Use this model to convert Urdu script text into Roman Urdu, useful for:

• Messaging apps and keyboards where Urdu script isn't easily typed
• Accessibility tools
• Language learning aids
• Search/indexing systems that need Romanized text

Out-of-Scope Use

Not intended for general-purpose text generation, translation into English, or use with languages other than Urdu.

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_name = "ogx786/urdu-roman-transliterator-tiny-aya"
adapter_repo = "hina6198/urdu-roman-transliterator-lora"  # update with your username

tokenizer = AutoTokenizer.from_pretrained(adapter_repo)
base_model = AutoModelForCausalLM.from_pretrained(base_model_name, device_map="auto")
model = PeftModel.from_pretrained(base_model, adapter_repo)
model.eval()

def transliterate(urdu_text):
    prompt = f"""### Instruction:
Transliterate Urdu to Roman Urdu.

### Input:
{urdu_text}

### Response:
"""
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    outputs = model.generate(**inputs, max_new_tokens=100, do_sample=False)
    result = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return result.split("### Response:")[-1].strip()

print(transliterate("تم کیسے ہو؟"))

Training Details

Training Data

[Describe your dataset here — size, source, e.g. "~X,000 Urdu–Roman Urdu sentence pairs collected/curated from ..."]

Training Procedure

• Method: LoRA fine-tuning (PEFT)
• LoRA config: r=16, alpha=32, target modules: q_proj, v_proj, dropout=0.05
• Epochs: 3
• Batch size: 1 (effective batch size 4 via gradient accumulation)
• Learning rate: 2e-4
• Precision: fp16

Training Hyperparameters

• Training regime: fp16 mixed precision
• Trainable parameters: ~3.8M (0.11% of total 3.35B parameters)

Limitations

• May not generalize well to dialectal variations, slang, or code-mixed (Urdu-English) text.
• Trained on a relatively small dataset; performance on rare words or names may be inconsistent.
• Base model is small