Downloads · 30 days
4
8% of all-time downloads
sfulay/zephyr-7b-dpo-full-ultrabin-reward-scale-1-rpo
zephyr-7b-dpo-full-ultrabin-reward-scale-1-rpo is a machine learning model from sfulay. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
4
8% of all-time downloads
All-time downloads
52
Public
Parameters
7.2B
101 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors14.5 GB · 100%
From the Hugging Face model README
This model is a fine-tuned version of alignment-handbook/zephyr-7b-sft-full on an unknown dataset. It achieves the following results on the evaluation set:
More information needed
More information needed
More information needed
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.0497 | 0.1046 | 50 | 0.0471 | 0.0341 | -0.0639 | 0.6953 | 0.0980 | -269.0505 | -259.2180 | -2.5747 | -2.6118 |
| 0.0399 | 0.2092 | 100 | 0.0400 | -0.0674 | -0.3039 | 0.7656 | 0.2365 | -293.0492 | -269.3653 | -2.2263 | -2.2796 |
| 0.0384 | 0.3138 | 150 | 0.0368 | -0.1521 | -0.4051 | 0.7812 | 0.2530 | -303.1761 | -277.8396 | -2.4575 | -2.5017 |
| 0.0354 | 0.4184 | 200 | 0.0368 | -0.1608 | -0.4413 | 0.7812 | 0.2805 | -306.7949 | -278.7134 | -2.6355 | -2.6785 |
| 0.035 | 0.5230 | 250 | 0.0359 | -0.0276 | -0.3002 | 0.7812 | 0.2726 | -292.6817 | -265.3905 | -2.5364 | -2.5931 |
| 0.0336 | 0.6276 | 300 | 0.0351 | -0.1609 | -0.4489 | 0.7734 | 0.2880 | -307.5566 | -278.7195 | -2.3179 | -2.4060 |
| 0.0338 | 0.7322 | 350 | 0.0348 | -0.1145 | -0.3940 | 0.7695 | 0.2795 | -302.0604 | -274.0787 | -2.3603 | -2.4329 |
| 0.0352 | 0.8368 | 400 | 0.0345 | -0.1250 | -0.4112 | 0.7734 | 0.2863 | -303.7862 | -275.1277 | -2.4372 | -2.5111 |
| 0.0342 | 0.9414 | 450 | 0.0345 | -0.1240 | -0.4158 | 0.7734 | 0.2918 | -304.2409 | -275.0308 | -2.4195 | -2.5003 |