origin-task3-nllb-lora

Origin SciHigh 2026 Task 3 NLLB-600M LoRA

This repository contains a LoRA adapter for English-to-Bengali scientific-paper title translation. It is the Run 1 system submitted by team Origin to SciHigh 2026 Task 3.

The adapter must be loaded on top of facebook/nllb-200-distilled-600M at commit f8d333a098d19b4fd9a8b18f94170487ad3f821d; it is not a standalone full model.

Model details

Training data and procedure

The adapter was trained only on the 60 official training triplets from the SciHigh 2026 Task 3 SpringerSSAT-Tiny-Multilingual split. Each input was the English title and each target was its expert Bengali translation. No synthetic examples or test labels were used.

The 20 official validation examples were held out from gradient optimization and used once per epoch for checkpoint selection.

Validation results

Scores below were calculated on the 20 held-out validation titles. ROUGE-L uses whitespace-delimited Bengali tokens

Usage

pip install "transformers==4.53.2" "peft>=0.16,<0.18" \
  "sentencepiece>=0.2,<0.3" torch

import torch
from peft import PeftModel
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

adapter_id = "uthayamurthy/origin-task3-nllb-lora"
base_id = "facebook/nllb-200-distilled-600M"
base_revision = "f8d333a098d19b4fd9a8b18f94170487ad3f821d"
device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(
    adapter_id, src_lang="eng_Latn", tgt_lang="ben_Beng"
)
base_model = AutoModelForSeq2SeqLM.from_pretrained(
    base_id,
    revision=base_revision,
    torch_dtype=torch.bfloat16 if device == "cuda" else torch.float32,
)
model = PeftModel.from_pretrained(base_model, adapter_id).to(device).eval()

title = "A framework to measure microaggressions in the mathematics classroom"
inputs = tokenizer(title, return_tensors="pt").to(device)
with torch.inference_mode():
    tokens = model.generate(
        **inputs,
        forced_bos_token_id=tokenizer.convert_tokens_to_ids("ben_Beng"),
        num_beams=5,
        max_new_tokens=96,
    )
print(tokenizer.batch_decode(tokens, skip_special_tokens=True)[0])