Downloads · 30 days
4
15% of all-time downloads
llama_SFT_e1_DPO_e1
thorirhrafn/llama_SFT_e1_DPO_e1
llama_SFT_e1_DPO_e1 is a machine learning model from thorirhrafn. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as llama2.
should probably proofread and complete it, then remove this comment. --
All-time downloads
27
Public
Repo size
269 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors268 MB · 100%
At a glance
- Library
- peft
- License
- llama2
- Access
- Public
- Created
- May 15, 2024
- Updated
- May 15, 2024
- SHA
- 3a17db10
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
llama_SFT_e1_DPO_e1
This model is a fine-tuned version of meta-llama/Llama-2-7b-hf on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.1876
- Rewards/chosen: 0.3221
- Rewards/rejected: -1.3485
- Rewards/accuracies: 1.0
- Rewards/margins: 1.6706
- Logps/rejected: -199.1326
- Logps/chosen: -156.6435
- Logits/rejected: -1.0544
- Logits/chosen: -0.8650
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-06
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1
Training results
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.6715 | 0.1 | 25 | 0.6332 | 0.0386 | -0.0871 | 0.9333 | 0.1257 | -186.5185 | -159.4784 | -1.0533 | -0.8570 |
| 0.5507 | 0.2 | 50 | 0.5213 | 0.1021 | -0.2851 | 1.0 | 0.3872 | -188.4984 | -158.8435 | -1.0540 | -0.8579 |
| 0.4521 | 0.3 | 75 | 0.4180 | 0.1622 | -0.5141 | 1.0 | 0.6763 | -190.7885 | -158.2424 | -1.0548 | -0.8606 |
| 0.3675 | 0.4 | 100 | 0.3332 | 0.2182 | -0.7466 | 1.0 | 0.9647 | -193.1132 | -157.6828 | -1.0545 | -0.8611 |
| 0.3149 | 0.5 | 125 | 0.2724 | 0.2574 | -0.9589 | 1.0 | 1.2164 | -195.2370 | -157.2902 | -1.0544 | -0.8631 |
| 0.2486 | 0.6 | 150 | 0.2247 | 0.2948 | -1.1593 | 1.0 | 1.4541 | -197.2406 | -156.9163 | -1.0550 | -0.8663 |
| 0.2173 | 0.7 | 175 | 0.1966 | 0.3176 | -1.2962 | 1.0 | 1.6138 | -198.6099 | -156.6887 | -1.0553 | -0.8673 |
| 0.1971 | 0.79 | 200 | 0.1878 | 0.3231 | -1.3461 | 1.0 | 1.6692 | -199.1087 | -156.6337 | -1.0542 | -0.8665 |
| 0.1869 | 0.89 | 225 | 0.1869 | 0.3210 | -1.3535 | 1.0 | 1.6745 | -199.1825 | -156.6541 | -1.0546 | -0.8626 |
| 0.1911 | 0.99 | 250 | 0.1876 | 0.3221 | -1.3485 | 1.0 | 1.6706 | -199.1326 | -156.6435 | -1.0544 | -0.8650 |
Framework versions
- PEFT 0.8.2
- Transformers 4.38.1
- Pytorch 2.2.0+cu118
- Datasets 2.17.1
- Tokenizers 0.15.2
- Library
- peft
- License
- llama2
- Created
- May 15, 2024
- Updated
- May 15, 2024