Downloads · 30 days
5
16% of all-time downloads
model_hh_shp4_dpo1
guoyu-zhang/model_hh_shp4_dpo1
model_hh_shp4_dpo1 is a machine learning model from guoyu-zhang. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
should probably proofread and complete it, then remove this comment. --
All-time downloads
32
Public
Repo size
12.6 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors25.2 MB · 93%
At a glance
- Library
- peft
- Access
- Public
- Created
- Apr 16, 2024
- Updated
- Apr 16, 2024
- SHA
- 4a707f81
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
model_hh_shp4_dpo1
This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.4317
- Rewards/chosen: -6.5985
- Rewards/rejected: -6.9380
- Rewards/accuracies: 0.5200
- Rewards/margins: 0.3394
- Logps/rejected: -304.6761
- Logps/chosen: -294.0565
- Logits/rejected: -0.8001
- Logits/chosen: -0.7815
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 4
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 100
- training_steps: 1000
Training results
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.0606 | 2.67 | 100 | 0.9815 | -3.8073 | -4.0570 | 0.5700 | 0.2497 | -275.8660 | -266.1440 | -0.7815 | -0.7820 |
| 0.0015 | 5.33 | 200 | 1.4766 | -9.4439 | -10.0498 | 0.5200 | 0.6059 | -335.7948 | -322.5099 | -0.9100 | -0.8860 |
| 0.0001 | 8.0 | 300 | 1.3563 | -5.3778 | -5.6296 | 0.5 | 0.2518 | -291.5923 | -281.8489 | -0.8443 | -0.8261 |
| 0.0001 | 10.67 | 400 | 1.3918 | -6.1165 | -6.4338 | 0.5200 | 0.3173 | -299.6345 | -289.2365 | -0.8181 | -0.8000 |
| 0.0001 | 13.33 | 500 | 1.4173 | -6.3997 | -6.7201 | 0.5200 | 0.3204 | -302.4971 | -292.0679 | -0.8077 | -0.7892 |
| 0.0001 | 16.0 | 600 | 1.4263 | -6.5107 | -6.8464 | 0.5200 | 0.3357 | -303.7607 | -293.1784 | -0.8030 | -0.7843 |
| 0.0001 | 18.67 | 700 | 1.4319 | -6.5813 | -6.9163 | 0.5200 | 0.3350 | -304.4596 | -293.8844 | -0.8006 | -0.7822 |
| 0.0001 | 21.33 | 800 | 1.4301 | -6.5939 | -6.9333 | 0.5200 | 0.3394 | -304.6292 | -294.0101 | -0.7998 | -0.7812 |
| 0.0001 | 24.0 | 900 | 1.4316 | -6.6029 | -6.9432 | 0.5200 | 0.3403 | -304.7287 | -294.0999 | -0.7994 | -0.7806 |
| 0.0001 | 26.67 | 1000 | 1.4317 | -6.5985 | -6.9380 | 0.5200 | 0.3394 | -304.6761 | -294.0565 | -0.8001 | -0.7815 |
Framework versions
- PEFT 0.10.0
- Transformers 4.39.1
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
- Library
- peft
- Created
- Apr 16, 2024
- Updated
- Apr 16, 2024