Downloads · 30 days
5
9% of all-time downloads
Llama-2-7b-hf-DPO-Filtered-0.2-version-3
LBK95/Llama-2-7b-hf-DPO-Filtered-0.2-version-3
Llama-2-7b-hf-DPO-Filtered-0.2-version-3 is a machine learning model from LBK95. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as llama2.
should probably proofread and complete it, then remove this comment. --
All-time downloads
57
Public
Repo size
2.4 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors604 MB · 100%
At a glance
- Library
- peft
- License
- llama2
- Access
- Public
- Created
- May 12, 2024
- Updated
- May 12, 2024
- SHA
- bf8bba86
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
Llama-2-7b-hf-DPO-Filtered-0.2-version-3
This model is a fine-tuned version of meta-llama/Llama-2-7b-hf on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.8860
- Rewards/chosen: -2.2683
- Rewards/rejected: -2.6831
- Rewards/accuracies: 0.5500
- Rewards/margins: 0.4148
- Logps/rejected: -83.1208
- Logps/chosen: -77.0439
- Logits/rejected: -1.3980
- Logits/chosen: -1.4004
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 10
- num_epochs: 3
Training results
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.6488 | 0.2994 | 72 | 0.6678 | -0.0112 | -0.0724 | 0.6000 | 0.0613 | -57.0147 | -54.4726 | -0.5812 | -0.5741 |
| 0.6 | 0.5988 | 144 | 0.6316 | -0.4870 | -0.6800 | 0.6000 | 0.1930 | -63.0904 | -59.2310 | -0.6746 | -0.6711 |
| 0.5876 | 0.8981 | 216 | 0.6931 | -0.4396 | -0.5539 | 0.5 | 0.1143 | -61.8289 | -58.7568 | -0.5937 | -0.5907 |
| 0.4949 | 1.1975 | 288 | 0.7890 | -0.7079 | -0.9614 | 0.6500 | 0.2535 | -65.9037 | -61.4400 | -0.8747 | -0.8740 |
| 0.565 | 1.4969 | 360 | 0.9088 | -1.6793 | -1.8869 | 0.5500 | 0.2077 | -75.1596 | -71.1538 | -1.2245 | -1.2255 |
| 0.283 | 1.7963 | 432 | 0.8288 | -1.8095 | -2.1999 | 0.6000 | 0.3905 | -78.2897 | -72.4555 | -1.2749 | -1.2766 |
| 0.1794 | 2.0956 | 504 | 0.8811 | -1.8931 | -2.2411 | 0.5500 | 0.3480 | -78.7009 | -73.2920 | -1.3148 | -1.3161 |
| 0.3907 | 2.3950 | 576 | 0.8772 | -2.2014 | -2.6107 | 0.5500 | 0.4093 | -82.3973 | -76.375 | -1.4219 | -1.4232 |
| 0.0225 | 2.6944 | 648 | 0.8843 | -2.2655 | -2.6784 | 0.5500 | 0.4129 | -83.0741 | -77.0161 | -1.3961 | -1.3981 |
| 0.2077 | 2.9938 | 720 | 0.8860 | -2.2683 | -2.6831 | 0.5500 | 0.4148 | -83.1208 | -77.0439 | -1.3980 | -1.4004 |
Framework versions
- PEFT 0.10.0
- Transformers 4.40.2
- Pytorch 2.3.0+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1
- Library
- peft
- License
- llama2
- Created
- May 12, 2024
- Updated
- May 12, 2024