abo-yoruba-llama-3.2-3b

Àbò — a Yorùbá fine-tune of Llama 3.2 3B

A LoRA fine-tune of Llama-3.2-3B-Instruct adapted for Yorùbá, trained with Adaption AutoScientist on a 100% human-sourced, contamination-proof corpus (see the dataset card). Built for the AutoScientist challenge — language track.

What it's good at

• Diacritics restoration — adding correct tone marks and sub-dots to bare Yorùbá text.
• Formal / news translation — English ↔ Yorùbá on formal text (its training domain).
• Producing fluent, correctly-accented Yorùbá where the base model defaults to English or garbles the marks.

Results (measured — real numbers only)

On 100 held-out examples from the task distribution, judged by Gemini 3.1 Pro (win rate vs. the base model):

Honest trade-off: on a broader multi-language benchmark the win rate went the other way (≈57 → 43). The model specialized into Yorùbá at the cost of general multilingual breadth — expected for a Yorùbá-track adaptation, and worth stating plainly.

Training

• Method: supervised fine-tuning, LoRA (r=32, α=64, all-linear), train_on_inputs=false.
• Data: 19,997 human rows, SHA-256 a7116920… (byte-identical to the released dataset — this guarantees the released dataset is what the model trained on).
• Platform: Adaption AutoScientist (4× H100).

Intended use

Research and non-commercial use for Yorùbá NLP: diacritics restoration, formal translation, and Yorùbá QA. It is a specialist, not a general chat assistant.

Limitations & risks

• Not a conversational chatbot. The corpus is news/formal, so casual/everyday Yorùbá (e.g. greetings) is out-of-domain and can be fluent-but-inaccurate.
• 3B parameters — limited capacity.
• Inherits base-model and source-data biases. Outputs should be checked by a native speaker before any downstream use.

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "meta-llama/Llama-3.2-3B-Instruct"   # or an equivalent mirror
tok = AutoTokenizer.from_pretrained("<this-repo>")
model = PeftModel.from_pretrained(AutoModelForCausalLM.from_pretrained(base), "<this-repo>")

msg = [{"role": "user", "content": "Restore the correct Yoruba diacritics (tone marks and dots) in this text:\n\nBawo ni, se alaafia ni?"}]
ids = tok.apply_chat_template(msg, add_generation_prompt=True, return_tensors="pt", return_dict=True)
print(tok.decode(model.generate(**ids, max_new_tokens=120, repetition_penalty=1.3, no_repeat_ngram_size=3)[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))

License

CC-BY-NC-4.0 (inherits the dataset's most restrictive source term). Base model under the Llama 3.2 Community License.