Downloads · 30 days
15
22% of all-time downloads
tinyllama_moe_dpo_ultrafeedback_epochs5
ondevicellm/tinyllama_moe_dpo_ultrafeedback_epochs5
tinyllama_moe_dpo_ultrafeedback_epochs5 is a text generation model from ondevicellm. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
All-time downloads
67
Public
Parameters
6.4B
12.9 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors12.9 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- mixtral
- Access
- Public
- Created
- Jan 16, 2024
- Updated
- Jan 17, 2024
- SHA
- 6233371b
Try a prompt
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
tinyllama_moe_dpo_ultrafeedback_epochs5
This model is a fine-tuned version of ondevicellm/tinyllama_moe_sft_ultrachat_epochs3 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.5698
- Rewards/chosen: -1.5249
- Rewards/rejected: -2.1850
- Rewards/accuracies: 0.7460
- Rewards/margins: 0.6601
- Logps/rejected: -525.1185
- Logps/chosen: -501.3176
- Logits/rejected: -1.7144
- Logits/chosen: -1.8206
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-07
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 2
- total_train_batch_size: 64
- total_eval_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 96
- num_epochs: 5
Training results
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.6916 | 0.1 | 100 | 0.6914 | 0.0044 | -0.0007 | 0.6290 | 0.0050 | -306.6867 | -348.3935 | -2.7893 | -2.8529 |
| 0.6838 | 0.21 | 200 | 0.6833 | 0.0222 | -0.0020 | 0.6548 | 0.0242 | -306.8183 | -346.6077 | -2.7745 | -2.8394 |
| 0.6683 | 0.31 | 300 | 0.6720 | 0.0087 | -0.0449 | 0.6647 | 0.0536 | -311.1083 | -347.9552 | -2.7457 | -2.8123 |
| 0.655 | 0.42 | 400 | 0.6583 | -0.0568 | -0.1516 | 0.6766 | 0.0948 | -321.7766 | -354.5066 | -2.6922 | -2.7610 |
| 0.6435 | 0.52 | 500 | 0.6453 | -0.1710 | -0.3165 | 0.6706 | 0.1455 | -338.2649 | -365.9251 | -2.6457 | -2.7160 |
| 0.641 | 0.63 | 600 | 0.6366 | -0.2120 | -0.3985 | 0.6825 | 0.1865 | -346.4684 | -370.0310 | -2.5893 | -2.6615 |
| 0.6207 | 0.73 | 700 | 0.6319 | -0.2615 | -0.4812 | 0.6706 | 0.2197 | -354.7403 | -374.9797 | -2.5381 | -2.6120 |
| 0.6025 | 0.84 | 800 | 0.6249 | -0.3306 | -0.5888 | 0.6964 | 0.2583 | -365.5025 | -381.8849 | -2.4644 | -2.5413 |
| 0.6317 | 0.94 | 900 | 0.6185 | -0.5597 | -0.8426 | 0.7063 | 0.2829 | -390.8784 | -404.7987 | -2.4027 | -2.4814 |
| 0.6087 | 1.05 | 1000 | 0.6137 | -0.5045 | -0.8126 | 0.7004 | 0.3081 | -387.8767 | -399.2817 | -2.3842 | -2.4637 |
| 0.5993 | 1.15 | 1100 | 0.6077 | -0.6040 | -0.9415 | 0.7044 | 0.3375 | -400.7663 | -409.2302 | -2.3436 | -2.4254 |
| 0.5628 | 1.26 | 1200 | 0.6026 | -0.8401 | -1.2238 | 0.7103 | 0.3837 | -429.0004 | -432.8431 | -2.2635 | -2.3475 |
| 0.5856 | 1.36 | 1300 | 0.5971 | -0.7421 | -1.1421 | 0.7242 | 0.3999 | -420.8279 | -423.0439 | -2.2233 | -2.3091 |
| 0.5672 | 1.47 | 1400 | 0.5930 | -0.7829 | -1.2202 | 0.7143 | 0.4373 | -428.6362 | -427.1146 | -2.1938 | -2.2804 |
| 0.5536 | 1.57 | 1500 | 0.5872 | -0.8347 | -1.2945 | 0.7202 | 0.4599 | -436.0717 | -432.2956 | -2.1433 | -2.2324 |
| 0.5669 | 1.67 | 1600 | 0.5858 | -0.7867 | -1.2636 | 0.7163 | 0.4769 | -432.9818 | -427.4996 | -2.1168 | -2.2065 |
| 0.5312 | 1.78 | 1700 | 0.5831 | -0.9925 | -1.4919 | 0.7262 | 0.4994 | -455.8103 | -448.0764 | -2.0492 | -2.1424 |
| 0.5596 | 1.88 | 1800 | 0.5798 | -1.0023 | -1.5297 | 0.7361 | 0.5274 | -459.5894 | -449.0625 | -2.0168 | -2.1124 |
| 0.5489 | 1.99 | 1900 | 0.5813 | -0.8832 | -1.3904 | 0.7202 | 0.5072 | -445.6621 | -437.1509 | -2.0039 | -2.0990 |
| 0.5327 | 2.09 | 2000 | 0.5795 | -0.9218 | -1.4418 | 0.7242 | 0.5200 | -450.7982 | -441.0125 | -1.9626 | -2.0594 |
| 0.5225 | 2.2 | 2100 | 0.5779 | -1.1696 | -1.7317 | 0.7401 | 0.5621 | -479.7868 | -465.7886 | -1.9207 | -2.0189 |
| 0.5085 | 2.3 | 2200 | 0.5769 | -1.1637 | -1.7425 | 0.7321 | 0.5789 | -480.8718 | -465.1949 | -1.8892 | -1.9891 |
| 0.5255 | 2.41 | 2300 | 0.5770 | -1.2191 | -1.7925 | 0.7341 | 0.5734 | -485.8650 | -470.7390 | -1.8632 | -1.9614 |
| 0.5116 | 2.51 | 2400 | 0.5742 | -1.1139 | -1.6936 | 0.7381 | 0.5798 | -475.9834 | -460.2151 | -1.8765 | -1.9749 |
| 0.5279 | 2.62 | 2500 | 0.5741 | -1.1556 | -1.7455 | 0.7361 | 0.5899 | -481.1734 | -464.3928 | -1.8664 | -1.9651 |
| 0.4795 | 2.72 | 2600 | 0.5745 | -1.1558 | -1.7459 | 0.7321 | 0.5900 | -481.2056 | -464.4143 | -1.8345 | -1.9355 |
| 0.5217 | 2.83 | 2700 | 0.5699 | -1.3475 | -1.9659 | 0.7440 | 0.6184 | -503.2092 | -483.5756 | -1.7956 | -1.8981 |
| 0.4945 | 2.93 | 2800 | 0.5699 | -1.3594 | -1.9727 | 0.7381 | 0.6132 | -503.8864 | -484.7731 | -1.8126 | -1.9141 |
| 0.477 | 3.04 | 2900 | 0.5721 | -1.3627 | -1.9877 | 0.7361 | 0.6250 | -505.3890 | -485.0972 | -1.7954 | -1.8980 |
| 0.4754 | 3.14 | 3000 | 0.5729 | -1.4117 | -2.0575 | 0.7321 | 0.6458 | -512.3726 | -490.0027 | -1.7473 | -1.8516 |
| 0.4696 | 3.24 | 3100 | 0.5708 | -1.5486 | -2.1921 | 0.7282 | 0.6435 | -525.8281 | -503.6902 | -1.7318 | -1.8363 |
| 0.4804 | 3.35 | 3200 | 0.5730 | -1.5037 | -2.1632 | 0.7321 | 0.6595 | -522.9344 | -499.1950 | -1.7097 | -1.8163 |
| 0.483 | 3.45 | 3300 | 0.5706 | -1.5793 | -2.2451 | 0.7302 | 0.6658 | -531.1252 | -506.7562 | -1.7082 | -1.8147 |
| 0.4791 | 3.56 | 3400 | 0.5723 | -1.4505 | -2.1095 | 0.7262 | 0.6590 | -517.5656 | -493.8777 | -1.7222 | -1.8274 |
| 0.4866 | 3.66 | 3500 | 0.5713 | -1.5091 | -2.1642 | 0.7381 | 0.6551 | -523.0358 | -499.7364 | -1.7191 | -1.8243 |
| 0.4651 | 3.77 | 3600 | 0.5731 | -1.4577 | -2.1177 | 0.7401 | 0.6600 | -518.3928 | -494.6030 | -1.7161 | -1.8217 |
| 0.483 | 3.87 | 3700 | 0.5708 | -1.4280 | -2.0759 | 0.7361 | 0.6479 | -514.2116 | -491.6330 | -1.7275 | -1.8325 |
| 0.4859 | 3.98 | 3800 | 0.5698 | -1.5249 | -2.1850 | 0.7460 | 0.6601 | -525.1185 | -501.3176 | -1.7144 | -1.8206 |
| 0.476 | 4.08 | 3900 | 0.5701 | -1.5060 | -2.1668 | 0.7440 | 0.6608 | -523.2975 | -499.4326 | -1.7157 | -1.8219 |
| 0.4553 | 4.19 | 4000 | 0.5705 | -1.5415 | -2.2042 | 0.7361 | 0.6626 | -527.0359 | -502.9834 | -1.7053 | -1.8120 |
| 0.4864 | 4.29 | 4100 | 0.5721 | -1.5310 | -2.1997 | 0.7381 | 0.6687 | -526.5859 | -501.9312 | -1.6982 | -1.8054 |
| 0.4402 | 4.4 | 4200 | 0.5720 | -1.5402 | -2.2110 | 0.7401 | 0.6708 | -527.7231 | -502.8538 | -1.6937 | -1.8008 |
| 0.4619 | 4.5 | 4300 | 0.5712 | -1.5462 | -2.2169 | 0.7361 | 0.6706 | -528.3046 | -503.4531 | -1.6931 | -1.8004 |
| 0.4421 | 4.6 | 4400 | 0.5710 | -1.5628 | -2.2323 | 0.7381 | 0.6695 | -529.8489 | -505.1078 | -1.6915 | -1.7989 |
| 0.4518 | 4.71 | 4500 | 0.5711 | -1.5704 | -2.2407 | 0.7361 | 0.6703 | -530.6893 | -505.8743 | -1.6913 | -1.7985 |
| 0.4508 | 4.81 | 4600 | 0.5715 | -1.5739 | -2.2436 | 0.7381 | 0.6697 | -530.9782 | -506.2146 | -1.6908 | -1.7981 |
| 0.484 | 4.92 | 4700 | 0.5716 | -1.5737 | -2.2419 | 0.7321 | 0.6682 | -530.8127 | -506.2016 | -1.6901 | -1.7976 |
Framework versions
- Transformers 4.36.2
- Pytorch 2.1.2+cu118
- Datasets 2.14.6
- Tokenizers 0.15.0
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- apache-2.0
- Created
- Jan 16, 2024
- Updated
- Jan 17, 2024