X-ALMA-13B-Nepali-Legal-Translator

X-ALMA-13B English-to-Nepali Legal Translator (PRO)

An experimental English-to-Nepali translation model fine-tuned on administrative, legal, and general-domain parallel datasets. The model utilizes a QLoRA adapter built on top of the instruction-tuned translation foundation haoranxu/X-ALMA-13B-Group7.

Disclaimer

[!WARNING] This model is in an experimental/draft stage (v1.0-draft). While it displays high fluency in general translations and most legal texts, it still exhibits minor inaccuracies under complex, number-heavy legal documents (e.g., occasional number mismatches or phrasing inconsistencies). It is NOT ready for production legal translation without strict human-in-the-loop validation. Users must verify all outputs, particularly numbers, dates, names, and key legal conditions.

---

Model Description

• Base Model: haoranxu/X-ALMA-13B-Group7 (13B Parameters)
• Adapter Type: QLoRA (NF4 4-bit quantization, targeted on q_proj and v_proj)
• Rank (r): 16
• Alpha (lora_alpha): 32
• Target Dataset: Parallel English-Nepali PRO translation dataset (~495K training, 5K validation samples)

---

Training History & Convergence

The model was fine-tuned for 8,000 steps with an effective batch size of 16. Validation loss steadily decreased, showing solid convergence:

---

Usage (Python API)

To use this adapter, it must be loaded on top of the base model haoranxu/X-ALMA-13B-Group7 (nested PEFT format):

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

model_id = "haoranxu/X-ALMA-13B-Group7"
adapter_id = "sandipghimire/X-ALMA-13B-Nepali-Legal-Translator"

# Load Tokenizer
tokenizer = AutoTokenizer.from_pretrained(adapter_id)

# Load Base Model
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16
)
base_model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=bnb_config,
    device_map="auto"
)

# Load Adapter
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

# Translate
prompt_content = "Translate this from English to Nepali:\nEnglish: Hello, how are you?\nNepali:"
messages = [{"role": "user", "content": prompt_content}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([prompt], return_tensors="pt").to("cuda")

with torch.no_grad():
    generated_ids = model.generate(**inputs, max_new_tokens=256, do_sample=False)
translation = tokenizer.decode(generated_ids[0][len(inputs.input_ids[0]):], skip_special_tokens=True).strip()
print(translation)