mayan-mt5-qeqchi-adapter

Mayan-mT5: Q'eqchi' Translation Adapter (Baseline STL)

This repository contains the Single-Task Learning (STL) LoRA adapter weights (Checkpoint 20000) for bidirectional machine translation between English/Spanish and Q'eqchi'. It is designed to be used in conjunction with the foundational google/mt5-base model.

Status: Phase 1 complete. This model was trained purely on a synthetic corpus to establish a syntactic scaffold and has been accepted to TSD 2026.

Repository Cross-Links

• Training Code & Script (GitHub): achulzhanov/mayan-mt5
• Training Dataset (Hugging Face): achulz/mayan-mt5-qeqchi-dataset
• Multi-Task Learning (MTL) Variant: achulz/mayan-mt5-qeqchi-mtl-adapter

Usage and Inference

Because this is a PEFT/LoRA adapter, you must load the base mt5-base model first, then apply these weights.

Required Task Prefix The model expects a clean, standardized task prefix matching the format: translate [Source Language] to [Target Language]:

• Valid Languages: English, Spanish, Q'eqchi'

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from peft import PeftModel

base_model_id = "google/mt5-base"
peft_model_id = "achulz/mayan-mt5-qeqchi-adapter"

# 1. Load Tokenizer and Base Model
tokenizer = AutoTokenizer.from_pretrained(peft_model_id) # Uses local custom spiece.model
base_model = AutoModelForSeq2SeqLM.from_pretrained(base_model_id)

# 2. Load the LoRA Adapter
model = PeftModel.from_pretrained(base_model, peft_model_id)

# 3. Format the Input with the correct Task Prefix
task_prefix = "translate English to Q'eqchi': "
source_text = "The dog is sleeping in the house."
input_text = task_prefix + source_text

# 4. Tokenize and Generate
inputs = tokenizer(input_text, return_tensors="pt", max_length=128, truncation=True)
outputs = model.generate(**inputs, max_new_tokens=128)
translation = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(translation)

Limitations & Bias

This baseline adapter was trained exclusively on a rule-based generated synthetic dataset. While it demonstrates strong foundational grammar mapping and structural acquisition (recovering Verb-Object-Subject word order), it suffers from a structural-semantic gap when exposed to natural out-of-domain language. It is intended as a grammatical primer for Phase 2 Curriculum Training refinement, not as a standalone, production-ready translation tool.