Downloads · 30 days
4
14% of all-time downloads
model_hh_usp2_dpo5
guoyu-zhang/model_hh_usp2_dpo5
model_hh_usp2_dpo5 is a machine learning model from guoyu-zhang. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
should probably proofread and complete it, then remove this comment. --
All-time downloads
28
Public
Repo size
12.6 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors25.2 MB · 93%
At a glance
- Library
- peft
- Access
- Public
- Created
- Apr 16, 2024
- Updated
- Apr 16, 2024
- SHA
- 4920ae2a
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
model_hh_usp2_dpo5
This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 2.3713
- Rewards/chosen: -6.8988
- Rewards/rejected: -10.4722
- Rewards/accuracies: 0.5700
- Rewards/margins: 3.5734
- Logps/rejected: -136.2930
- Logps/chosen: -125.5019
- Logits/rejected: -0.5150
- Logits/chosen: -0.4766
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 4
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 100
- training_steps: 1000
Training results
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.1335 | 2.67 | 100 | 1.8678 | -12.7163 | -15.5473 | 0.6400 | 2.8311 | -146.4432 | -137.1367 | -0.5513 | -0.5250 |
| 0.0599 | 5.33 | 200 | 1.9478 | -7.7572 | -11.3317 | 0.6800 | 3.5745 | -138.0119 | -127.2186 | -0.6179 | -0.5774 |
| 0.0178 | 8.0 | 300 | 2.5739 | -15.2954 | -20.0626 | 0.6400 | 4.7672 | -155.4737 | -142.2950 | -0.8409 | -0.7974 |
| 0.0001 | 10.67 | 400 | 2.3449 | -6.7892 | -10.3661 | 0.5800 | 3.5769 | -136.0806 | -125.2826 | -0.5079 | -0.4695 |
| 0.0 | 13.33 | 500 | 2.3597 | -6.8546 | -10.4323 | 0.5800 | 3.5776 | -136.2131 | -125.4135 | -0.5123 | -0.4736 |
| 0.0 | 16.0 | 600 | 2.3541 | -6.8691 | -10.4554 | 0.5800 | 3.5863 | -136.2593 | -125.4424 | -0.5143 | -0.4755 |
| 0.0 | 18.67 | 700 | 2.3647 | -6.8932 | -10.4651 | 0.5800 | 3.5719 | -136.2787 | -125.4907 | -0.5147 | -0.4761 |
| 0.0 | 21.33 | 800 | 2.3640 | -6.8827 | -10.4754 | 0.5800 | 3.5928 | -136.2994 | -125.4696 | -0.5155 | -0.4766 |
| 0.0 | 24.0 | 900 | 2.3566 | -6.8881 | -10.4744 | 0.5800 | 3.5863 | -136.2972 | -125.4803 | -0.5148 | -0.4762 |
| 0.0 | 26.67 | 1000 | 2.3713 | -6.8988 | -10.4722 | 0.5700 | 3.5734 | -136.2930 | -125.5019 | -0.5150 | -0.4766 |
Framework versions
- PEFT 0.10.0
- Transformers 4.39.3
- Pytorch 2.2.2+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
- Library
- peft
- Created
- Apr 16, 2024
- Updated
- Apr 16, 2024