Downloads · 30 days
4
5% of all-time downloads
sfulay/zephyr-7b-dpo-full-ultrabin-reward-scale-1
zephyr-7b-dpo-full-ultrabin-reward-scale-1 is a machine learning model from sfulay. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
4
5% of all-time downloads
All-time downloads
80
Public
Parameters
7.2B
72.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors14.5 GB · 100%
From the Hugging Face model README
This model is a fine-tuned version of alignment-handbook/zephyr-7b-sft-full on the HuggingFaceH4/ultrafeedback_binarized dataset. It achieves the following results on the evaluation set:
More information needed
More information needed
More information needed
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.6644 | 0.1046 | 50 | 0.6488 | 0.0260 | -0.0889 | 0.7109 | 0.1149 | -271.5553 | -260.0293 | -2.5794 | -2.6162 |
| 0.5487 | 0.2092 | 100 | 0.5902 | -1.2155 | -1.9393 | 0.7227 | 0.7239 | -456.5951 | -384.1769 | 0.4524 | 0.1576 |
| 0.5256 | 0.3138 | 150 | 0.5479 | -1.0848 | -2.0806 | 0.7344 | 0.9958 | -470.7209 | -371.1125 | 1.5020 | 0.9718 |
| 0.5215 | 0.4184 | 200 | 0.5366 | -1.2109 | -2.2989 | 0.7656 | 1.0880 | -492.5526 | -383.7176 | 1.7166 | 1.0342 |
| 0.5123 | 0.5230 | 250 | 0.5273 | -1.0432 | -2.1107 | 0.7617 | 1.0675 | -473.7350 | -366.9494 | 0.9023 | 0.2939 |
| 0.4963 | 0.6276 | 300 | 0.5263 | -1.4844 | -2.7207 | 0.7734 | 1.2363 | -534.7312 | -411.0707 | 1.6647 | 0.8936 |
| 0.4915 | 0.7322 | 350 | 0.5202 | -1.5546 | -2.7469 | 0.7773 | 1.1924 | -537.3561 | -418.0878 | 1.2880 | 0.4607 |
| 0.5018 | 0.8368 | 400 | 0.5239 | -1.6225 | -2.8974 | 0.7656 | 1.2750 | -552.4055 | -424.8771 | 1.5572 | 0.6801 |
| 0.497 | 0.9414 | 450 | 0.5231 | -1.5820 | -2.8565 | 0.7695 | 1.2746 | -548.3168 | -420.8284 | 1.4463 | 0.5693 |