Downloads · 30 days
4
2% of all-time downloads
Jongbin-kr/llama-3.1-8b-instruct-4x1-moe_SNI-c_2461-lora-moe
llama-3.1-8b-instruct-4x1-moe_SNI-c_2461-lora-moe is a text generation model from Jongbin-kr. Use it when you need the model to write or continue text. It is set up for peft.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
4
2% of all-time downloads
All-time downloads
201
Public
Repo size
15.5 GB
Likes
0
Public
Click a slice to open those files.
.safetensors454 MB · 96%
From the Hugging Face model README
This model is a fine-tuned version of Jongbin-kr/llama-3.1-8b-instruct-4x1-moe on an unknown dataset. It achieves the following results on the evaluation set:
More information needed
More information needed
More information needed
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | Supervised Loss |
|---|---|---|---|---|
| 2.6054 | 0.0300 | 100 | 1.5550 | 1.2605 |
| 2.1675 | 0.0599 | 200 | 1.4097 | 0.8473 |
| 2.1097 | 0.0899 | 300 | 1.3688 | 0.7445 |
| 2.0516 | 0.1198 | 400 | 1.3483 | 0.5749 |
| 2.0457 | 0.1498 | 500 | 1.3317 | 0.5804 |
| 1.8742 | 0.1797 | 600 | 1.3158 | 0.4153 |
| 2.1618 | 0.2097 | 700 | 1.2849 | 0.4308 |
| 1.8731 | 0.2396 | 800 | 1.2773 | 0.4498 |
| 1.9175 | 0.2696 | 900 | 1.2710 | 0.4065 |
| 2.0463 | 0.2995 | 1000 | 1.2458 | 0.3431 |
| 1.6688 | 0.3295 | 1100 | 1.2275 | 0.3168 |
| 2.0435 | 0.3594 | 1200 | 1.2253 | 0.3322 |
| 2.0428 | 0.3894 | 1300 | 1.2399 | 0.2872 |
| 1.6628 | 0.4194 | 1400 | 1.2061 | 0.2940 |
| 1.7990 | 0.4493 | 1500 | 1.2606 | 0.2549 |
| 1.7976 | 0.4793 | 1600 | 1.1935 | 0.2688 |
| 1.9108 | 0.5092 | 1700 | 1.1903 | 0.3102 |
| 1.8318 | 0.5392 | 1800 | 1.1907 | 0.2640 |
| 1.9078 | 0.5691 | 1900 | 1.1814 | 0.3138 |
| 1.8186 | 0.5991 | 2000 | 1.1747 | 0.2957 |
| 1.8156 | 0.6290 | 2100 | 1.1687 | 0.2719 |
| 1.8816 | 0.6590 | 2200 | 1.1667 | 0.2944 |
| 2.0673 | 0.6889 | 2300 | 1.1597 | 0.2806 |
| 2.0624 | 0.7189 | 2400 | 1.1592 | 0.2784 |
| 1.7396 | 0.7488 | 2500 | 1.1623 | 0.2671 |
| 1.8832 | 0.7788 | 2600 | 1.1528 | 0.2726 |
| 1.8187 | 0.8087 | 2700 | 1.1516 | 0.2650 |
| 1.7010 | 0.8387 | 2800 | 1.1508 | 0.2651 |
| 1.7230 | 0.8687 | 2900 | 1.1457 | 0.2705 |
| 1.6429 | 0.8986 | 3000 | 1.1464 | 0.2671 |
| 1.6427 | 0.9286 | 3100 | 1.1455 | 0.2682 |
| 1.9052 | 0.9585 | 3200 | 1.1469 | 0.2682 |
| 1.5876 | 0.9885 | 3300 | 1.1457 | 0.2680 |
| 1.5975 | 1.0 | 3339 | 1.1453 | 0.2678 |