Qwen1.5-7B-Translator

Qwen1.5-7B-Translator

Model Details

Model Description

• Finetuned from model : Qwen1.5-7B-Chat

This model is a SFT(Supervised Fine-Tuning) version of Qwen1.5-7B-Chat, trained on an additional Chinese-English document translation data set. After fine-tuning, the model's accuracy in translating professional terms in physics, chemistry, biology and other disciplines has been greatly enhanced, and can even be comparable to professional translation tools such as deepL.

We used LLaMA-Factory for fine-tuning. In order to avoid catastrophic forgetting after model fine-tuning, we tried to diversify the fine-tuning instructions and incorporated a small number of multi-round long conversations as the training set.

More details about training and evaluation can be found at: https://github.com/DeyangKong/TransLingoBridge

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Direct Use

The instruction format of this model is qwen.

This format is available as a chat template via the apply_chat_template() method:

import torch
from transformers import AutoModelForCausalLM
from transformers import AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("DeyangKong/Qwen1.5-7B-Translator", torch_dtype=torch.float16, device_map="auto", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("DeyangKong/Qwen1.5-7B-Translator", trust_remote_code=True)

sentence = "Photocatalytic reduction of nitrate with TiO2 progressed with varied kinetics and to different products based on applied wavelengths."
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": f"As a professional translator, you're requested to convey the following English text into Chinese, ensuring the exactitude of specialized terminology:\n{sentence}"}
]
encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")
generated_ids = model.generate(encodeds, max_new_tokens=1024)
decoded = tokenizer.batch_decode(generated_ids)
print(decoded[0])

We used a variety of prompt words to build the data set during fine-tuning. One of them is used in the above code. Some prompt words are as follows:

Given your proficiency in English-Chinese translation, please ensure the following text is translated with precision, especially concerning technical and specialized terms:
Utilize your translation skills to accurately transform the following English sentences into Chinese, paying special attention to the proper translation of technical terms:
As a professional translator, you're requested to convey the following English text into Chinese, ensuring the exactitude of specialized terminology:
In your role as a language translation authority, accurately interpret the next English segment into Chinese, with emphasis on the precise translation of terminologies:
You are tasked with translating the subsequent English text into Chinese, as a translation specialist, ensuring that all technical terms are accurately translated:
Drawing upon your translation expertise, please meticulously translate the following English text into Chinese, ensuring the fidelity of specialized terms:

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

We compared the translation capabilities of deepL, Alma and Qwen1.5-7B-Translator on 99 professional documents involving several fields such as computer, chemistry, biology, materials science, etc., and used GPT4 as the judge for ranking.

The results It is shown that our model is only slightly inferior to deepL and far superior to Alma.

<img src="https://cdn-uploads.huggingface.co/production/uploads/65a0aade5fafc248c2156e95/zF9DRrkHV9Hj9hHxRs0rK.png">

At the same time, our model also solves the problem of Alma’s translation errors on long texts of more than 3000 characters.

More details about training and evaluation can be found at: https://github.com/DeyangKong/TransLingoBridge