DeepSeek-R1-Distill-Qwen-14B-Multilingual

lightblue/DeepSeek-R1-Distill-Qwen-14B-Multilingual

<div style="width: 100%; height: 160px; display: flex; align-items: center; justify-content: center; border: 8px solid black; font-size: 120px; font-weight: bold; text-align: center; color: #438db8, font-family: 'Helvetica Neue', sans-serif;"> <span style="color: #438db8;">R1</span> &nbsp; <span style="color: blue;">m</span> <span style="color: green;">u</span> <span style="color: purple;">l</span> <span style="color: yellow;">t</span> <span style="color: pink;">i</span> <span style="color: cyan;">l</span> <span style="color: magenta;">i</span> <span style="color: lime;">n</span> <span style="color: teal;">g</span> </div>

This is a Deepseek distill finetune trained on multilingual Chain-of-Thought (CoT). When this model is prompted in a language, it will both think and respond in that language, unlike the original R1 which will often think in either Chinese or English. This will make the outputs of these AIs more understandable and explainable to a wider audience. Hopefully this will be useful to the AI community, particularly those developing for languages aside from English and Chinese.

This model is a multilingual fine-tuned version of deepseek-ai/DeepSeek-R1-Distill-Qwen-14B.

Other fine-tuned versions of this model can be found in our collection, here.

This model was trained was trained using our lightblue/reasoning-multilingual-R1-Llama-70B-train dataset for ~10 minutes on the 8 x L20 instance (ecs.gn8is-8x.32xlarge) on Alibaba Cloud.

How to use

When using these models, we recommend using a sampling temperature of between 0.5-0.7, as per the original distilled R1 models.

Additionally, we have observed that the model sometimes tends to repeat for more niche languages, so we also recommend setting repetition_penalty to 1.1, or higher if the model repeats itself when processing your prompts.

We include scripts to use this model in vLLM:

<ul> <li><b>vLLM</b>

Install vLLM using pip install vllm.

<details open> <summary>Show vLLM code</summary>

from vllm import LLM, SamplingParams

llm = LLM(
    model="lightblue/DeepSeek-R1-Distill-Qwen-7B-Multilingual",
    max_model_len=8_000
)

sampling_params = SamplingParams(
    temperature=0.5, 
    max_tokens=8_000
)

prompts = [
    """学校には1クラスにつき20人の生徒がおり、クラスは合計3つあります。
学校全体では男子と女子がそれぞれ50%ずついます。
1つ目のクラスには女子が15人、2つ目のクラスには女子が12人います。
3つ目のクラスには何人の男子がいますか？"""
]

conversations = [
    [{"role": "user", "content": x}] for x in prompts
]

outputs = llm.chat(conversations, sampling_params=sampling_params)

for output in outputs:
    print(output.outputs[0].text)

# <think>
# まず、学校の総生徒数を算出します。各クラスに20人の生徒があり、クラスは3つあるため、総生徒数は60人です。

# 次に、学校全体で男子と女子は同じ人数で分布しています。したがって、男子と女子各有30人。
...
# したがって、3つ目のクラスの男子数は20 - 3 = 17人です。
# </think>

# **解答：**

# 学校の総生徒数を算出します。
...
# **最終的な答え：**
# \[
# \boxed{17}
# \]

</details></li> </ul>

Evaluation

Through some quick evaluation of our own, we found this model can produce much correctly formatted and accurate results for higher resource languages, such as Japanese, English, German, than lower resource languages, such as Amharic or Lao.

We did a very quick evaluation of 5 questions with each dataset (written by me and translated by GPT4o Mini) on the lightblue/DeepSeek-R1-Distill-Qwen-7B-Multilingual model, and we find that the model is able to fairly reliably output the correct answers and in the correct language for a large variety of languages:

For this evaluation, a score of >=0.8 is good, as one of the questions was very hard. The language detection was done using pycld2 so errors may occur with the correct language being mistaken for another one.

• There was an error with Farsi detection (my own fault) so we do not report Farsi scores.

The evaluation code for this can be found here.

Training code

### model
model_name_or_path: deepseek-ai/DeepSeek-R1-Distill-Qwen-14B

### method
stage: sft
do_train: true
finetuning_type: full
deepspeed: /root/LLaMA-Factory/examples/deepspeed/ds_z3_config.json

### dataset
dataset: reasoning-multilingual-R1-Llama-70B-train
template: qwen
cutoff_len: 4096
overwrite_cache: true
preprocessing_num_workers: 16
packing: true

### output
output_dir: /root/train_outputs/DeepSeek-R1-Distill-Qwen-14B/reasoning-multilingual-R1-Llama-70B-train
logging_steps: 1
save_steps: 0.99999
plot_loss: true
overwrite_output_dir: true

### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 1
learning_rate: 1.0e-5
num_train_epochs: 1.0
lr_scheduler_type: cosine
warmup_ratio: 0.01
bf16: true
ddp_timeout: 180000000

### eval
val_size: 0.01
per_device_eval_batch_size: 1
eval_strategy: steps
eval_steps: 0.1

echo '{
  "reasoning-multilingual-R1-Llama-70B-train": {
    "hf_hub_url": "lightblue/reasoning-multilingual-R1-Llama-70B-train",
    "formatting": "sharegpt"
  }
}' > /root/LLaMA-Factory/data/dataset_info.json

# # 14B Llama
cd /root/LLaMA-Factory && llamafactory-cli train /root/reasoning_multilingual_train_14B.yaml
rm -r /root/train_outputs/DeepSeek-R1-Distill-Qwen-14B/reasoning-multilingual-R1-Llama-70B-train/checkpoint*
huggingface-cli upload lightblue/DeepSeek-R1-Distill-Qwen-14B-Multilingual /root/train_outputs/DeepSeek-R1-Distill-Qwen-14B/reasoning-multilingual-R1-Llama-70B-train

License

We share this model with the Apache 2.0 license.

Developed by

<a href="https://www.lightblue-tech.com"> <img src="https://www.lightblue-tech.com/wp-content/uploads/2023/08/color_%E6%A8%AA%E5%9E%8B-1536x469.png" alt="Lightblue technology logo" width="400"/> </a>

This model was trained by Peter Devine (ptrdvn) for Lightblue