Qwen2.5-7B-Instruct-preference

<!-- Provide a quick summary of what the model is/does. -->

Qwen2.5-7B-Instruct-preference

Model Description

<!-- Provide a longer summary of what this model is. -->

Qwen2.5-7B-Instruct-preference is a fine-tuned model based on Qwen/Qwen2.5-7B-Instruct. This model is fine-tuned on original dataset. The fine-tuned were carried out at a 1024 context length.

Benchmarking

The benchmark score is obtained using arena-hard-auto-multilingual.

Model Details

• Model size: 7B
• Context length: 1024
• Language: Japanese

Training Procudure

• learning_rate: 5e-6
• train_batch_size: 4
• eval_batch_size: 2
• gradient_accumulation_steps: 4
• lr_scheduler_type: cosine

Training Results