amestris-1b-dpo

Amestris-1B-DPO

amestris-1b-dpo is a parameter-efficient English-to-German machine-translation adapter for google/gemma-3-1b-it. It was trained with Direct Preference Optimization (DPO) on approximately 27,000 preference pairs selected to emphasize difficult translation cases. The preferred responses are reference-quality German translations; the rejected responses are lower-quality Gemma translations identified through the project’s quality-filtering pipeline.

This repository contains a non-merged PEFT/LoRA adapter, not a standalone full-precision model. The Gemma 3 1B instruction-tuned base model must therefore be loaded before attaching this adapter.

Model details

The research methodology, data-construction pipeline, and evaluation protocol are described in the project repository and the associated paper.

Quick start: run the model

1. Install dependencies

pip install -U "transformers>=4.50.0" "peft>=0.18.1" "accelerate>=1.0" torch

The base model is gated. Before running the code, accept Google’s Gemma terms on the base-model page and authenticate with a Hugging Face account that has access:

hf auth login

2. Load the adapter and translate

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE_MODEL_ID = "google/gemma-3-1b-it"
ADAPTER_ID = "gaokerena/amestris-1b-dpo"

# The project evaluation code uses the base-model tokenizer to avoid a
# tokenizer/adapter mismatch during decoding.
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL_ID)

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL_ID,
    device_map="auto",
    dtype="auto",
)
model = PeftModel.from_pretrained(base_model, ADAPTER_ID)
model.eval()

source_text = "The committee will publish its final report next Tuesday."
translation_prompt = """You are an expert professional translator from English to German.
Preserve all meaning, nuance, factual details, names, dates, and numbers.
Use natural, fluent, professional standard German.
Do not omit, summarize, add, or explain anything.
Return only the German translation.

Text to translate:
If the street is clear, the pedestrian obtains a green light immediately, if not, there is a delay of around 15 seconds.
"""

messages = [{"role": "user", "content": translation_prompt}]
formatted_prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(formatted_prompt, return_tensors="pt")
device = next(model.parameters()).device
inputs = {name: tensor.to(device) for name, tensor in inputs.items()}

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
        num_beams=4,
        early_stopping=True,
        repetition_penalty=1.05,
        no_repeat_ngram_size=3,
    )

generated_ids = output_ids[0, inputs["input_ids"].shape[1]:]
translation = tokenizer.decode(generated_ids, skip_special_tokens=True).strip()
print(translation)

The example uses deterministic beam search to reflect the project’s translation-oriented evaluation setup. Generation parameters may be adjusted for latency or deployment constraints, but doing so can change evaluation outcomes.

Training methodology

The project constructs preference data from English–German translation examples by pairing a preferred reference translation with a rejected model output. Lower-quality candidate translations are isolated through automatic quality scoring and distribution-based filtering so that DPO receives a pronounced preference signal rather than a pair of near-equivalent translations.

The reported DPO configuration includes one epoch, a DPO temperature parameter of 0.1, a warmup ratio of 0.03, gradient accumulation over four steps, and LoRA with rank 32, alpha 32, dropout 0.05, and all linear projections targeted. The published adapter configuration identifies google/gemma-3-1b-it as the base and CAUSAL_LM as the PEFT task type.

Evaluation

The following English-to-German results are reported by the project on the WMT14 test split.

These are project-reported results, not independently reproduced during preparation of this model card. Comparisons should preserve the same prompt, decoding configuration, test split, preprocessing, and metric versions.

Intended use

The adapter is intended for research and experimentation in:

• English-to-German translation;
• preference optimization for machine translation;
• LoRA/PEFT ablations against the Gemma 3 1B baseline;
• comparisons among DPO, supervised fine-tuning, and sequential DPO→SFT.

It is not a certified translation service. Human review is recommended for legal, medical, financial, safety-critical, or publication-grade material.

Limitations and risks

• The model is specialized for English-to-German translation and has not been established as reliable for other directions or languages.
• Its 1B-parameter base limits capacity on long, highly technical, or context-dependent inputs.
• Project evaluation identified occasional corruption of names, dates, and numerical values, as well as rare hallucinations, sentence-alignment errors, and malformed generations.
• Automatic metrics do not fully measure factual fidelity, terminology consistency, cultural appropriateness, or downstream harm.
• The model may inherit biases and safety limitations from its base model and training data.
• Adapter use remains subject to the Gemma license and access conditions.

Repository contents

• adapter_model.safetensors: LoRA adapter weights.
• adapter_config.json: PEFT configuration identifying the base model and target modules.
• tokenizer.json, tokenizer_config.json, special_tokens_map.json, and chat_template.jinja: tokenizer-related artifacts included with the release.
• README.md: this model card.

Citation

If you use this model or the associated method, please cite:

@misc{ghassabi2026backtranslation,
  title         = {Backtranslation Augmented Direct Preference Optimization for Neural Machine Translation},
  author        = {Ghassabi, Mehrdad and Rajabi, Spehr and Baradaran Kashani, Hamidreza and Hakim, Sadra and Keivandarian, Mahshid and Jahani Bahnamiri, Amirhossein},
  year          = {2026},
  eprint        = {2604.25702},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL}
}

Acknowledgments

This checkpoint was developed as part of the Amestris research project by Mehrdad Ghassabi, Sepehr Rajabi, Hamidreza Baradaran Kashani, Sadra Hakim, Mahshid Keivandarian, and Amirhossein Jahani Bahnamiri.