en-dhivehi-qwen3-4b-lora-paragraph

Qwen3-4B English→Dhivehi (paragraph-level)

Qwen3-4B (decoder-only) + LoRA adapter for English→Dhivehi translation, trained on the multi-granularity Dhivehi–English parallel corpus (gated).

Qwen-based paragraph model — higher quality, heavier. For a lighter, faster option, see the mT5 paragraph variant. Handles both single sentences and multi-sentence paragraphs.

Scores (chrF / chrF++ / BLEU)

chrF is the metric to trust for Thaana; BLEU is unreliable (word segmentation / morphology).

Example

Input (en): While this is a 7.6 percent increase compared to the same period last year, an average of 7,778 tourists visit the Maldives daily.

Output (dv): މިދިޔަ އަހަރުގެ މި މުއްދަތާ ބަލާއިރު، މިއީ 7.6 ޕަސެންޓުގެ ކުރިއެރުމެއް ކަމަށްވާއިރު، ދުވާލަކު އެވްރެޖްކޮށް 7،778 ފަތުރުވެރިން ރާއްޖެއަށް ޒިޔާރަތްކުރެއެވެ.

Multi-sentence input (en): Nashidha Mohamed is a Maldivian film star. She has acted in many different Maldivian films. In addition to this, audiences have seen Naakko's talent in various video songs and TV drama series. Regarding her personal life, Nashidha Mohamed is an actress born in HDh. Kulhudhuffushi. She has now started a married life. Entry into the film industry: Nashidha Mohamed entered the film industry through a video song performed with film star Ahmed Asim. After that, she has acted in various dramas, films, and series.

Output (dv): ނާޝިދާ މުހައްމަދު އަކީ ދިވެހި ފިލްމީ ތަރިއެކެވެ. ތަފާތު ގިނަ ދިވެހި ފިލްމްތަކުގައި ބައިވެރި ވެފައި ވެއެވެ. މީގެ އިތުރުން، ތަފާތު ވީޑިޔޯ ލަވަތަކާއި، ޓީވީ ޑްރާމާ ސިލްސިލާ ތަކުން ނާއްކޮގެ ހުނަރު ބެލުންތެރިންނަށް ފެނިގެން ގޮސްފައި ވެއެވެ. އަމިއްލަ ދިރިއުޅުން ނާޝިދާ މުހައްމަދު އަކީ ހދ. ކުޅުދުއްފުއްޓަށް އުފަން އެކްޓްރެސް އެކެވެ. މިހާރު ޒަވާޖީ ދިރިއުޅުމެއް ފަށާފައި ވެއެވެ. ފިލްމީ ދާއިރާ އަށް ނިކުތުން ނާޝިދާ މުހައްމަދު ފިލްމީ ދާއިރާ އަށް ނިކުތީ ފިލްމީ ތަރި އަހުމަދު އާސިމް އާއެކު ކުޅުނު ވީޑިޔޯ ލަވަޔަކުންނެވެ. އޭގެ ފަހުން ތަފާތު ޑްރާމާ ، ފިލްމް، ސިލްސިލާ ތަކުގައި ހަރަކާތްތެރި ވެފައި ވެއެވެ.

Real held-out sample and this model's own output.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base, adapter = "Qwen/Qwen3-4B", "Neobe/en-dhivehi-qwen3-4b-lora-paragraph"
tok = AutoTokenizer.from_pretrained(adapter)
model = PeftModel.from_pretrained(
    AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.bfloat16, device_map="cuda"),
    adapter).eval()

src = "The President of the Maldives met with the cabinet today."
msgs = [{"role":"user","content":
    "Translate the following English text to Dhivehi. Output only the translation, "
    f"no explanations.\n\nEnglish: {src}\nDhivehi:"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inp = tok(prompt, return_tensors="pt", truncation=True, max_length=3072).to("cuda")
out = model.generate(**inp, max_new_tokens=1024, num_beams=1, repetition_penalty=1.15, no_repeat_ngram_size=0, length_penalty=1.0, do_sample=False)
print(tok.decode(out[0][inp["input_ids"].shape[1]:], skip_special_tokens=True))

~9–10 GB VRAM (bf16). Qwen tokenizes Thaana at ~1.8 tokens/char — don't over-truncate the input.

Training

Base Qwen/Qwen3-4B; LoRA r=16, α=32, targets q/k/v/o+gate/up/down; bf16; adamw_torch LR 2e-4 cosine; max_length 1536; 1 epoch; effective batch ~32; gradient checkpointing.

Limitations

Domain = Maldivian news / press / Wikipedia; technical or informal English is out of distribution. Non-human references are machine-generated (distillation).

Citation

@misc{neobe_en_dhivehi_qwen3_4b_lora_paragraph_2026,
  title  = {Qwen3-4B English→Dhivehi (paragraph-level)},
  author = {Neobe},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/Neobe/en-dhivehi-qwen3-4b-lora-paragraph}}
}