GaMS-9B-SFT-Translator-DPO

Model Card for GaMS-DPO-Translator

GaMS-Beta/GaMS-9B-SFT-Translator-DPO is a fine-tuned version of GaMS-9B-SFT-Translator. Direct Preference Optimization (DPO) was performed on the original model. The learning dataset was synthetially generated by using GaMS-9B-SFT-Translator and EuroLLM-9B-Instruct.

Basic information

• Developed by: team of researchers at the University of Ljubljana, Faculty for Computer and Information Science. Team members: Dario Vajda, Domen Vreš and Marko Robnik-Šikonja.
• Languages: Slovene, English (primary), Croatian, Bosnian and Serbian (secondary). The model might also work for other languages supported by Gemma 2, even though it was not continually pretrained on them.
• Base model: cjvt/GaMS-9B-Instruct
• License: Gemma

Usage

The model can be run through pipeline API using the following code:

from transformers import pipeline

model_id = "GaMS-Beta/GaMS-9B-SFT-Translator-DPO"

pline = pipeline(
    "text-generation",
    model=model_id,
    device_map="cuda" # replace with "mps" to run on a Mac device
)

# Example of response generation
message = [{"role": "user", "content": "Prevedi naslednje angleško besedilo v slovenščino.\nToday is a nice day."}]
response = pline(message, max_new_tokens=512)
print("Translation:", response[0]["generated_text"][-1]["content"])

For multi GPU inference set the device_map to auto:

from transformers import pipeline

model_id = "GaMS-Beta/GaMS-9B-SFT-Translator-DPO"

pline = pipeline(
    "text-generation",
    model=model_id,
    device_map="auto"
)

# Example of response generation
message = [{"role": "user", "content": "Prevedi naslednje angleško besedilo v slovenščino.\nToday is a nice day."}]
response = pline(message, max_new_tokens=512)
print("Model's response:", response[0]["generated_text"][-1]["content"])

# Example of conversation chain
new_message = response[0]["generated_text"]
new_message.append({"role": "user", "content": "Lahko bolj podrobno opišeš ta dogodek?"})
response = pline(new_message, max_new_tokens=1024)
print("Model's response:", response[0]["generated_text"][-1]["content"])

Data

Data for fine-tuning the original model was acquired by translating a large corpora of wikipedia articles, ccnews articles, bookcorpus texts and english conversational datasets by two models(GaMS-9B-SFT-Translator and EuroLLM-9B-Instruct) which were then ranked by some automatic metrics for translation quality and reliability.

Training

The model was trained on the Vega HPC

Evaluation

The model was evaluated by our custom script on three types of data. The results are show in the following table.

Note - the evaluation script and evaluation data can be found in this github repo under the data_pipeline folder. See the README for more detailed instructions.

Citation

If you found this project useful in your work, please cite our paper with the following BibTeX citation:

@misc{vajda2025improvingllmsmachinetranslation,
      title={Improving LLMs for Machine Translation Using Synthetic Preference Data}, 
      author={Dario Vajda and Domen Vreš and Marko Robnik-Šikonja},
      year={2025},
      eprint={2508.14951},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2508.14951}, 
}