wmt26-lt3-sorbian

WMT26 Limited-Resources LLM, Sorbian track (Ghent University, LT3)

Submission repository for the WMT26 "LLMs with Limited Resources" shared task, Sorbian track (LT3). All five tasks (MT, QA, spell-check, grammar-check, math-reasoning) are produced by one QLoRA-tuned Qwen 3.5 2B (r256, merged, parameter count unchanged) plus inference strategies.

Contents: five submitted systems over three weight sets

Plain and composed variants share weights (they differ only in the inference strategy), so the five uploaded systems reduce to three merged weight sets, one per subfolder.

Primary system is LT3-FullStack-DevTransfer, weights in LT3-DevTransfer/. Each subfolder is a complete HF model (config, tokenizer, model.safetensors of about 3.76 GB). The value of generation_config.eos_token_id is [248046, 248044], and both ids are required.

System summary

A single QLoRA-tuned Qwen 3.5 2B serves all five Sorbian tasks. Training combines the organiser data with public German-Sorbian parallel corpora (WMT20-22), back-translation of monolingual Sorbian with our own model, and calibrated synthetic task data (typo injection for SC, distribution-matched morphological-paradigm corruptions via soblex and dsb-spell for GC, machine-translated public maths seeds GSM8K, MATH and MathQA with PolyMath excluded, and cloze QA). Retrieval-augmented inference is used per the organisers' RAG clarification: half of the training examples carry a fuzzy-retrieved exemplar, and at inference MT retrieves k=3 exemplars from a real and back-translated translation memory that is fully enumerated, hashed, and deduplicated against all test sources (no web access). GC additionally consults corpus bigram-attestation statistics.

How the model is used: generate versus score

The frozen model plays two roles. For MT and MR it is a generator that produces the output text. For QA, SC and GC it is a scorer that ranks candidate answers or corrections by the likelihood it assigns them.

The FullStack systems add light task logic around this (option assignment for QA, a paradigm and corpus-witness arbiter for GC, dictionary-constrained correction for SC, exemplar retrieval for MT, and a retry ladder for MR). The Plain systems use the model output directly.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
sub = "LT3-DevTransfer"   # PRIMARY. Others: LT3-EarlyStop, LT3-HeldOutDev
tok = AutoTokenizer.from_pretrained("<this-repo>", subfolder=sub)
model = AutoModelForCausalLM.from_pretrained("<this-repo>", subfolder=sub, torch_dtype="bfloat16")

The full five-task inference pipeline (prompts, the inference strategies, per-system flags) will be made available in a public GitHub repository as part of the system-description paper.

Compliance

Base is Qwen 3.5 2B (at most 2B parameters, unchanged parameter count). One model per submission. All data is public or licensed and MANIFESTed, PolyMath is excluded, and there is zero test material in training or retrieval (dedup proof in the code repository). A full reproducibility pack (environment, one-pass commands, per-file hashes, and data-regeneration recipes) accompanies the system-description paper.

Weights are derived from Qwen 3.5 2B and inherit its model license.