Downloads · 30 days
13
17% of all-time downloads
YuchenLi01/genSoftQwen2.5MathRM72Bth0.5pair4NoGT_1.5B_dpo_ebs32_lr1e-06_beta1.5_epoch2.0_42
genSoftQwen2.5MathRM72Bth0.5pair4NoGT_1.5B_dpo_ebs32_lr1e-06_beta1.5_epoch2.0_42 is a text generation model from YuchenLi01. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
13
17% of all-time downloads
All-time downloads
77
Public
Parameters
1.5B
21.6 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors3.1 GB · 99%
From the Hugging Face model README
This model is a fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct on the YuchenLi01/MATH_Qwen2.5-1.5BInstruct_Soft_DPO_Qwen2.5MathRM72B_th0.5_pair4NoGT dataset. It achieves the following results on the evaluation set:
More information needed
More information needed
More information needed
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.7816 | 0.0287 | 20 | 0.7757 | -0.0482 | 0.0303 | 0.4146 | -0.0785 | -47.4995 | -37.3227 | -2.1582 | -2.2975 |
| 0.693 | 0.0573 | 40 | 0.7598 | -0.0512 | -0.0445 | 0.4817 | -0.0067 | -47.5494 | -37.3247 | -2.1585 | -2.2985 |
| 0.7015 | 0.0860 | 60 | 0.7594 | -0.0390 | -0.0771 | 0.4756 | 0.0382 | -47.5711 | -37.3165 | -2.1640 | -2.3048 |
| 0.7306 | 0.1146 | 80 | 0.7255 | -0.0526 | -0.1225 | 0.5427 | 0.0699 | -47.6014 | -37.3257 | -2.1663 | -2.3084 |
| 0.6579 | 0.1433 | 100 | 0.6830 | 0.0964 | -0.0695 | 0.5427 | 0.1659 | -47.5661 | -37.2263 | -2.1778 | -2.3213 |
| 0.7196 | 0.1719 | 120 | 0.6601 | 0.0655 | -0.2701 | 0.6159 | 0.3357 | -47.6998 | -37.2469 | -2.1715 | -2.3163 |
| 0.6426 | 0.2006 | 140 | 0.6524 | 0.2038 | -0.3342 | 0.6951 | 0.5379 | -47.7425 | -37.1547 | -2.1807 | -2.3270 |
| 0.6997 | 0.2292 | 160 | 0.6410 | -0.1270 | -0.7998 | 0.6768 | 0.6728 | -48.0529 | -37.3752 | -2.1657 | -2.3145 |
| 0.6034 | 0.2579 | 180 | 0.6420 | 0.2730 | -0.4736 | 0.7195 | 0.7465 | -47.8354 | -37.1086 | -2.1906 | -2.3405 |
| 0.8915 | 0.2865 | 200 | 0.6473 | 0.3913 | -0.4153 | 0.7195 | 0.8066 | -47.7966 | -37.0297 | -2.1958 | -2.3475 |
| 0.6017 | 0.3152 | 220 | 0.6686 | -0.5213 | -1.3297 | 0.7073 | 0.8085 | -48.4062 | -37.6381 | -2.1418 | -2.2931 |
| 0.4561 | 0.3438 | 240 | 0.6759 | -0.6318 | -1.7183 | 0.7012 | 1.0865 | -48.6652 | -37.7118 | -2.1472 | -2.3008 |
| 0.9137 | 0.3725 | 260 | 0.6450 | 0.3486 | -0.5980 | 0.6829 | 0.9466 | -47.9184 | -37.0582 | -2.1831 | -2.3381 |
| 0.6471 | 0.4011 | 280 | 0.6573 | 0.1015 | -0.8122 | 0.6585 | 0.9137 | -48.0612 | -37.2229 | -2.1665 | -2.3222 |
| 0.9354 | 0.4298 | 300 | 0.6536 | -0.3367 | -1.3807 | 0.6890 | 1.0440 | -48.4402 | -37.5150 | -2.1563 | -2.3133 |
| 0.7952 | 0.4585 | 320 | 0.6550 | -0.1077 | -1.3223 | 0.7195 | 1.2146 | -48.4012 | -37.3624 | -2.1681 | -2.3244 |
| 0.6926 | 0.4871 | 340 | 0.6457 | -0.0733 | -1.2992 | 0.7561 | 1.2258 | -48.3858 | -37.3395 | -2.1588 | -2.3151 |
| 0.7714 | 0.5158 | 360 | 0.6543 | 0.1237 | -1.2069 | 0.6829 | 1.3306 | -48.3243 | -37.2081 | -2.1710 | -2.3283 |
| 0.5304 | 0.5444 | 380 | 0.6706 | -0.2258 | -1.6375 | 0.7195 | 1.4117 | -48.6114 | -37.4411 | -2.1719 | -2.3305 |
| 0.7861 | 0.5731 | 400 | 0.6817 | 0.3212 | -1.0317 | 0.7073 | 1.3529 | -48.2075 | -37.0765 | -2.1981 | -2.3558 |
| 0.9531 | 0.6017 | 420 | 0.6756 | -0.2541 | -1.7245 | 0.7012 | 1.4704 | -48.6694 | -37.4600 | -2.1704 | -2.3283 |
| 0.8798 | 0.6304 | 440 | 0.6629 | 0.2772 | -1.0388 | 0.6829 | 1.3160 | -48.2123 | -37.1058 | -2.1921 | -2.3490 |
| 0.5759 | 0.6590 | 460 | 0.6818 | -0.0844 | -1.3346 | 0.6890 | 1.2503 | -48.4095 | -37.3468 | -2.1781 | -2.3361 |
| 0.9174 | 0.6877 | 480 | 0.6880 | -0.3364 | -1.6457 | 0.6951 | 1.3093 | -48.6169 | -37.5149 | -2.1686 | -2.3269 |
| 0.66 | 0.7163 | 500 | 0.6909 | -0.6942 | -2.1095 | 0.7378 | 1.4153 | -48.9260 | -37.7534 | -2.1601 | -2.3192 |
| 0.7585 | 0.7450 | 520 | 0.7019 | -0.0684 | -1.3202 | 0.6890 | 1.2518 | -48.3998 | -37.3362 | -2.1851 | -2.3445 |
| 0.6394 | 0.7736 | 540 | 0.7072 | -0.6344 | -1.8517 | 0.7256 | 1.2173 | -48.7542 | -37.7135 | -2.1517 | -2.3117 |
| 1.0728 | 0.8023 | 560 | 0.7158 | -1.0004 | -2.3608 | 0.6890 | 1.3604 | -49.0936 | -37.9575 | -2.1380 | -2.2978 |
| 0.8325 | 0.8309 | 580 | 0.7257 | -1.3725 | -2.6897 | 0.6646 | 1.3172 | -49.3128 | -38.2056 | -2.1227 | -2.2837 |
| 0.7326 | 0.8596 | 600 | 0.7272 | -1.1372 | -2.5377 | 0.6098 | 1.4006 | -49.2115 | -38.0487 | -2.1380 | -2.2997 |
| 0.6805 | 0.8883 | 620 | 0.7230 | -0.6642 | -2.1547 | 0.6463 | 1.4904 | -48.9562 | -37.7334 | -2.1600 | -2.3208 |
| 0.9775 | 0.9169 | 640 | 0.7088 | -0.5655 | -1.9725 | 0.6524 | 1.4071 | -48.8347 | -37.6675 | -2.1745 | -2.3367 |
| 0.6044 | 0.9456 | 660 | 0.7091 | -0.1881 | -1.6725 | 0.6768 | 1.4844 | -48.6347 | -37.4159 | -2.1837 | -2.3453 |
| 0.7081 | 0.9742 | 680 | 0.7162 | -0.4524 | -2.0292 | 0.6768 | 1.5769 | -48.8725 | -37.5921 | -2.1699 | -2.3324 |
| 0.1596 | 1.0029 | 700 | 0.6988 | -0.7874 | -2.3268 | 0.6829 | 1.5394 | -49.0709 | -37.8155 | -2.1546 | -2.3173 |
| 0.1757 | 1.0315 | 720 | 0.7079 | -0.6389 | -2.2087 | 0.6768 | 1.5697 | -48.9921 | -37.7165 | -2.1632 | -2.3271 |
| 0.1835 | 1.0602 | 740 | 0.6932 | -0.3258 | -2.0826 | 0.7012 | 1.7568 | -48.9081 | -37.5078 | -2.1746 | -2.3386 |
| 0.0827 | 1.0888 | 760 | 0.7007 | -0.2270 | -2.0654 | 0.7012 | 1.8385 | -48.8967 | -37.4419 | -2.1828 | -2.3479 |
| 0.2117 | 1.1175 | 780 | 0.6920 | -0.3278 | -2.2424 | 0.7073 | 1.9146 | -49.0147 | -37.5091 | -2.1811 | -2.3474 |
| 0.3506 | 1.1461 | 800 | 0.6959 | -0.1403 | -2.0081 | 0.6829 | 1.8677 | -48.8584 | -37.3841 | -2.1984 | -2.3649 |
| 0.5385 | 1.1748 | 820 | 0.7008 | -0.1942 | -2.0555 | 0.6951 | 1.8613 | -48.8900 | -37.4200 | -2.1979 | -2.3650 |
| 0.125 | 1.2034 | 840 | 0.7167 | -0.6307 | -2.4205 | 0.6890 | 1.7898 | -49.1334 | -37.7110 | -2.1763 | -2.3431 |
| 0.0442 | 1.2321 | 860 | 0.7246 | -0.4946 | -2.3734 | 0.6890 | 1.8788 | -49.1020 | -37.6203 | -2.1866 | -2.3544 |
| 0.5252 | 1.2607 | 880 | 0.7204 | -0.6199 | -2.5875 | 0.6707 | 1.9675 | -49.2447 | -37.7039 | -2.1799 | -2.3472 |
| 0.0731 | 1.2894 | 900 | 0.7097 | -0.6569 | -2.6145 | 0.6707 | 1.9576 | -49.2627 | -37.7285 | -2.1822 | -2.3499 |
| 0.0469 | 1.3181 | 920 | 0.7019 | -0.7703 | -2.8326 | 0.6890 | 2.0623 | -49.4081 | -37.8041 | -2.1762 | -2.3426 |
| 0.0882 | 1.3467 | 940 | 0.7028 | -0.5575 | -2.6672 | 0.7073 | 2.1097 | -49.2978 | -37.6622 | -2.1874 | -2.3553 |
| 0.2356 | 1.3754 | 960 | 0.7031 | -0.5834 | -2.6197 | 0.6585 | 2.0362 | -49.2661 | -37.6795 | -2.1941 | -2.3621 |
| 0.0858 | 1.4040 | 980 | 0.7183 | -0.5751 | -2.7300 | 0.7195 | 2.1549 | -49.3397 | -37.6739 | -2.1925 | -2.3615 |
| 0.174 | 1.4327 | 1000 | 0.7130 | -0.6201 | -2.8080 | 0.6890 | 2.1879 | -49.3917 | -37.7040 | -2.1900 | -2.3591 |
| 0.0354 | 1.4613 | 1020 | 0.7249 | -0.5798 | -2.8008 | 0.7134 | 2.2210 | -49.3869 | -37.6771 | -2.1905 | -2.3594 |
| 0.058 | 1.4900 | 1040 | 0.7192 | -0.4973 | -2.6283 | 0.6890 | 2.1310 | -49.2719 | -37.6221 | -2.2004 | -2.3696 |
| 0.2637 | 1.5186 | 1060 | 0.7112 | -0.5327 | -2.7055 | 0.7256 | 2.1727 | -49.3233 | -37.6457 | -2.1998 | -2.3688 |
| 0.0883 | 1.5473 | 1080 | 0.7194 | -0.4208 | -2.6713 | 0.6829 | 2.2505 | -49.3006 | -37.5711 | -2.2038 | -2.3738 |
| 0.0831 | 1.5759 | 1100 | 0.7070 | -0.4275 | -2.7299 | 0.7134 | 2.3023 | -49.3396 | -37.5756 | -2.2008 | -2.3703 |
| 0.147 | 1.6046 | 1120 | 0.7104 | -0.3600 | -2.6528 | 0.7012 | 2.2928 | -49.2883 | -37.5306 | -2.2030 | -2.3725 |
| 0.1492 | 1.6332 | 1140 | 0.7176 | -0.4854 | -2.6823 | 0.6829 | 2.1969 | -49.3079 | -37.6142 | -2.1993 | -2.3687 |
| 0.0542 | 1.6619 | 1160 | 0.7267 | -0.5271 | -2.7202 | 0.7256 | 2.1932 | -49.3332 | -37.6419 | -2.1991 | -2.3685 |
| 0.1355 | 1.6905 | 1180 | 0.7069 | -0.4707 | -2.7593 | 0.7073 | 2.2886 | -49.3592 | -37.6044 | -2.2002 | -2.3703 |
| 0.0901 | 1.7192 | 1200 | 0.7210 | -0.4801 | -2.7593 | 0.7073 | 2.2792 | -49.3592 | -37.6106 | -2.1973 | -2.3666 |
| 0.205 | 1.7479 | 1220 | 0.7141 | -0.5845 | -2.7396 | 0.7195 | 2.1551 | -49.3461 | -37.6802 | -2.1960 | -2.3653 |
| 0.0402 | 1.7765 | 1240 | 0.7136 | -0.4569 | -2.6785 | 0.7378 | 2.2217 | -49.3054 | -37.5951 | -2.2030 | -2.3728 |
| 0.1517 | 1.8052 | 1260 | 0.7051 | -0.4537 | -2.7524 | 0.7134 | 2.2986 | -49.3546 | -37.5931 | -2.1995 | -2.3700 |
| 0.1625 | 1.8338 | 1280 | 0.7029 | -0.4230 | -2.6815 | 0.7317 | 2.2585 | -49.3074 | -37.5726 | -2.2005 | -2.3700 |
| 0.122 | 1.8625 | 1300 | 0.7011 | -0.4832 | -2.7072 | 0.7012 | 2.2241 | -49.3245 | -37.6127 | -2.1956 | -2.3646 |
| 0.2707 | 1.8911 | 1320 | 0.7185 | -0.4797 | -2.6968 | 0.7012 | 2.2170 | -49.3176 | -37.6104 | -2.1993 | -2.3695 |
| 0.0793 | 1.9198 | 1340 | 0.7152 | -0.5183 | -2.6727 | 0.6890 | 2.1544 | -49.3015 | -37.6361 | -2.1974 | -2.3668 |
| 0.2281 | 1.9484 | 1360 | 0.7147 | -0.5079 | -2.7271 | 0.7256 | 2.2192 | -49.3377 | -37.6292 | -2.1910 | -2.3593 |
| 0.2948 | 1.9771 | 1380 | 0.7152 | -0.5212 | -2.6645 | 0.6890 | 2.1433 | -49.2960 | -37.6380 | -2.1950 | -2.3636 |