Downloads · 30 days
10
17% of all-time downloads
generatedSoftQwen2.5MathRM72Bth0.5LessNoGT_Qwen2.5-1.5BInstruct_dpo_ebs32_lr5e-07_beta0.4_42
YuchenLi01/generatedSoftQwen2.5MathRM72Bth0.5LessNoGT_Qwen2.5-1.5BInstruct_dpo_ebs32_lr5e-07_beta0.4_42
generatedSoftQwen2.5MathRM72Bth0.5LessNoGT_Qwen2.5-1.5BInstruct_dpo_ebs32_lr5e-07_beta0.4_42 is a text generation model from YuchenLi01. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
All-time downloads
59
Public
Parameters
1.5B
3.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.1 GB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- qwen2
- Access
- Public
- Created
- Jun 6, 2025
- Updated
- Jun 6, 2025
- SHA
- 8b819df0
Try a prompt
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
generatedSoftQwen2.5MathRM72Bth0.5LessNoGT_Qwen2.5-1.5BInstruct_dpo_ebs32_lr5e-07_beta0.4_42
This model is a fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct on the YuchenLi01/MATH_Qwen2.5-1.5BInstruct_Soft_DPO_Qwen2.5MathRM72B_th0.5_LessNoGT dataset. It achieves the following results on the evaluation set:
- Loss: 0.6576
- Rewards/chosen: -0.0737
- Rewards/rejected: -0.1430
- Rewards/accuracies: 0.5625
- Rewards/margins: 0.0693
- Logps/rejected: -42.8545
- Logps/chosen: -34.0529
- Logits/rejected: -2.3327
- Logits/chosen: -2.4028
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-07
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- distributed_type: multi-GPU
- num_devices: 8
- total_train_batch_size: 32
- total_eval_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 1.0
Training results
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.7049 | 0.1010 | 20 | 0.6983 | 0.0197 | -0.0170 | 0.4583 | 0.0367 | -42.5396 | -33.8193 | -2.3275 | -2.3907 |
| 0.6884 | 0.2020 | 40 | 0.6859 | 0.0488 | 0.0167 | 0.5417 | 0.0322 | -42.4553 | -33.7465 | -2.3429 | -2.4077 |
| 0.6768 | 0.3030 | 60 | 0.6791 | 0.0474 | -0.0147 | 0.5625 | 0.0621 | -42.5337 | -33.7501 | -2.3422 | -2.4086 |
| 0.6723 | 0.4040 | 80 | 0.6712 | 0.0357 | -0.0341 | 0.6667 | 0.0698 | -42.5821 | -33.7793 | -2.3405 | -2.4086 |
| 0.6592 | 0.5051 | 100 | 0.6614 | -0.0258 | -0.0792 | 0.5 | 0.0534 | -42.6949 | -33.9329 | -2.3344 | -2.4030 |
| 0.6216 | 0.6061 | 120 | 0.6614 | -0.0390 | -0.1156 | 0.5417 | 0.0766 | -42.7859 | -33.9661 | -2.3301 | -2.3989 |
| 0.6186 | 0.7071 | 140 | 0.6585 | -0.0434 | -0.1329 | 0.5417 | 0.0894 | -42.8291 | -33.9772 | -2.3373 | -2.4074 |
| 0.6329 | 0.8081 | 160 | 0.6568 | -0.0560 | -0.1271 | 0.4583 | 0.0711 | -42.8148 | -34.0085 | -2.3341 | -2.4037 |
| 0.596 | 0.9091 | 180 | 0.6499 | -0.0386 | -0.1249 | 0.5417 | 0.0863 | -42.8092 | -33.9649 | -2.3344 | -2.4046 |
Framework versions
- Transformers 4.45.2
- Pytorch 2.5.1+cu121
- Datasets 3.5.0
- Tokenizers 0.20.3
- Task
- Text Generation
- Library
- transformers
- Type
- qwen2
- License
- apache-2.0
- Created
- Jun 6, 2025
- Updated
- Jun 6, 2025