apertus-8b-prussian-youtube

apertus-8b-prussian-youtube

LoRA adapter for swiss-ai/Apertus-8B-Instruct-2509 that translates into reconstructed neo-Prussian.

Trained on ~3.7k sentence pairs harvested from Old-Prussian YouTube subtitle tracks (source languages English / Lithuanian / Latvian → Old Prussian), single direction (XX→PR), plain output. Trained with Apertus's native chat template, so the assistant turn ends on the real eos <|assistant_end|> (no stray markup in the output).

Prompt format

Instruction in the system role, source sentence as the only user content:

messages = [
    {"role": "system", "content": "Translate to reconstructed neo-prussian:"},
    {"role": "user", "content": "I go into the forest."},
]

The exact system prompt matters. Putting the instruction into the user turn (or omitting the system message) takes the model off-distribution.

Usage

Trained on an int8-quantized base; load the base in int8 for best fidelity (the LoRA deltas were tuned for that representation).

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base = "swiss-ai/Apertus-8B-Instruct-2509"
adapter = "strfry/apertus-8b-prussian-youtube"

tok = AutoTokenizer.from_pretrained(adapter)
model = AutoModelForCausalLM.from_pretrained(
    base, device_map="auto",
    quantization_config=BitsAndBytesConfig(load_in_8bit=True),
)
model = PeftModel.from_pretrained(model, adapter).eval()

messages = [
    {"role": "system", "content": "Translate to reconstructed neo-prussian:"},
    {"role": "user", "content": "Ich gehe in den Wald"},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True,
                                 return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
# -> As ēima en meddjan

Training

• Base: swiss-ai/Apertus-8B-Instruct-2509 (loaded int8 / bitsandbytes)
• Method: LoRA (r=8, α=32, dropout=0.05), targets q/k/v/o + gate/up/down
• 3 epochs, lr 2e-4 cosine; native chat template (tokenizer_default)
• Framework: Axolotl

Sibling adapter (EuroLLM-9B, Tatoeba): strfry/eurollm-9b-prussian-tatoeba.