Downloads · 30 days
4
13% of all-time downloads
llama_SFT_e1_DPO_e3
thorirhrafn/llama_SFT_e1_DPO_e3
llama_SFT_e1_DPO_e3 is a machine learning model from thorirhrafn. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as llama2.
should probably proofread and complete it, then remove this comment. --
All-time downloads
31
Public
Repo size
1.6 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors268 MB · 100%
At a glance
- Library
- peft
- License
- llama2
- Access
- Public
- Created
- May 15, 2024
- Updated
- May 22, 2024
- SHA
- ef6a610e
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
llama_SFT_e1_DPO_e3
This model is a fine-tuned version of meta-llama/Llama-2-7b-hf on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.0745
- Rewards/chosen: 0.4581
- Rewards/rejected: -2.2850
- Rewards/accuracies: 1.0
- Rewards/margins: 2.7431
- Logps/rejected: -231.3585
- Logps/chosen: -178.1707
- Logits/rejected: -1.0559
- Logits/chosen: -0.8886
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 7e-07
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3
Training results
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.6769 | 0.1 | 25 | 0.6597 | 0.0250 | -0.0447 | 0.8267 | 0.0696 | -208.9552 | -182.5023 | -1.0564 | -0.8828 |
| 0.6046 | 0.2 | 50 | 0.5848 | 0.0656 | -0.1679 | 0.9767 | 0.2335 | -210.1874 | -182.0954 | -1.0566 | -0.8835 |
| 0.5278 | 0.3 | 75 | 0.4947 | 0.1260 | -0.3302 | 1.0 | 0.4561 | -211.8100 | -181.4921 | -1.0569 | -0.8842 |
| 0.4449 | 0.4 | 100 | 0.4127 | 0.1764 | -0.5163 | 1.0 | 0.6927 | -213.6715 | -180.9882 | -1.0574 | -0.8860 |
| 0.3778 | 0.5 | 125 | 0.3368 | 0.2273 | -0.7228 | 1.0 | 0.9502 | -215.7366 | -180.4786 | -1.0575 | -0.8860 |
| 0.2947 | 0.6 | 150 | 0.2704 | 0.2755 | -0.9495 | 1.0 | 1.2250 | -218.0034 | -179.9972 | -1.0567 | -0.8893 |
| 0.2451 | 0.7 | 175 | 0.2164 | 0.3191 | -1.1794 | 1.0 | 1.4985 | -220.3023 | -179.5607 | -1.0570 | -0.8899 |
| 0.1913 | 0.79 | 200 | 0.1737 | 0.3571 | -1.4023 | 1.0 | 1.7594 | -222.5310 | -179.1809 | -1.0573 | -0.8904 |
| 0.1553 | 0.89 | 225 | 0.1412 | 0.3875 | -1.6187 | 1.0 | 2.0062 | -224.6958 | -178.8769 | -1.0564 | -0.8903 |
| 0.1265 | 0.99 | 250 | 0.1173 | 0.4068 | -1.8122 | 1.0 | 2.2190 | -226.6304 | -178.6835 | -1.0564 | -0.8882 |
| 0.1174 | 1.09 | 275 | 0.1029 | 0.4201 | -1.9500 | 1.0 | 2.3701 | -228.0080 | -178.5508 | -1.0555 | -0.8881 |
| 0.0915 | 1.19 | 300 | 0.0938 | 0.4300 | -2.0411 | 1.0 | 2.4711 | -228.9190 | -178.4516 | -1.0555 | -0.8891 |
| 0.0819 | 1.29 | 325 | 0.0880 | 0.4386 | -2.1112 | 1.0 | 2.5497 | -229.6201 | -178.3662 | -1.0554 | -0.8884 |
| 0.09 | 1.39 | 350 | 0.0838 | 0.4485 | -2.1597 | 1.0 | 2.6082 | -230.1051 | -178.2668 | -1.0556 | -0.8886 |
| 0.0865 | 1.49 | 375 | 0.0803 | 0.4551 | -2.2012 | 1.0 | 2.6562 | -230.5201 | -178.2012 | -1.0556 | -0.8904 |
| 0.0731 | 1.59 | 400 | 0.0787 | 0.4544 | -2.2264 | 1.0 | 2.6807 | -230.7719 | -178.2080 | -1.0555 | -0.8884 |
| 0.0734 | 1.69 | 425 | 0.0769 | 0.4580 | -2.2470 | 1.0 | 2.7050 | -230.9783 | -178.1717 | -1.0552 | -0.8884 |
| 0.0827 | 1.79 | 450 | 0.0763 | 0.4591 | -2.2582 | 1.0 | 2.7173 | -231.0906 | -178.1612 | -1.0555 | -0.8884 |
| 0.0784 | 1.89 | 475 | 0.0756 | 0.4545 | -2.2709 | 1.0 | 2.7253 | -231.2172 | -178.2073 | -1.0556 | -0.8886 |
| 0.0768 | 1.99 | 500 | 0.0751 | 0.4564 | -2.2804 | 1.0 | 2.7368 | -231.3123 | -178.1877 | -1.0556 | -0.8883 |
| 0.0696 | 2.09 | 525 | 0.0746 | 0.4607 | -2.2824 | 1.0 | 2.7431 | -231.3322 | -178.1449 | -1.0554 | -0.8901 |
| 0.0691 | 2.19 | 550 | 0.0743 | 0.4597 | -2.2852 | 1.0 | 2.7449 | -231.3599 | -178.1548 | -1.0557 | -0.8886 |
| 0.07 | 2.29 | 575 | 0.0747 | 0.4597 | -2.2807 | 1.0 | 2.7404 | -231.3157 | -178.1549 | -1.0559 | -0.8889 |
| 0.0754 | 2.38 | 600 | 0.0744 | 0.4599 | -2.2865 | 1.0 | 2.7463 | -231.3729 | -178.1530 | -1.0555 | -0.8903 |
| 0.0833 | 2.48 | 625 | 0.0743 | 0.4588 | -2.2878 | 1.0 | 2.7466 | -231.3862 | -178.1637 | -1.0557 | -0.8887 |
| 0.0761 | 2.58 | 650 | 0.0746 | 0.4583 | -2.2856 | 1.0 | 2.7440 | -231.3646 | -178.1684 | -1.0557 | -0.8889 |
| 0.0716 | 2.68 | 675 | 0.0745 | 0.4597 | -2.2866 | 1.0 | 2.7463 | -231.3745 | -178.1546 | -1.0558 | -0.8905 |
| 0.0755 | 2.78 | 700 | 0.0746 | 0.4592 | -2.2835 | 1.0 | 2.7426 | -231.3430 | -178.1601 | -1.0560 | -0.8889 |
| 0.063 | 2.88 | 725 | 0.0740 | 0.4618 | -2.2897 | 1.0 | 2.7516 | -231.4058 | -178.1337 | -1.0556 | -0.8884 |
| 0.0743 | 2.98 | 750 | 0.0745 | 0.4581 | -2.2850 | 1.0 | 2.7431 | -231.3585 | -178.1707 | -1.0559 | -0.8886 |
Framework versions
- PEFT 0.8.2
- Transformers 4.38.1
- Pytorch 2.2.0+cu118
- Datasets 2.17.1
- Tokenizers 0.15.2
- Library
- peft
- License
- llama2
- Created
- May 15, 2024
- Updated
- May 22, 2024