Downloads · 30 days
3
11% of all-time downloads
llama-7b_alpaca_l0.0002_64
alexander-hm/llama-7b_alpaca_l0.0002_64
llama-7b_alpaca_l0.0002_64 is a machine learning model from alexander-hm. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as other.
should probably proofread and complete it, then remove this comment. --
All-time downloads
28
Public
Repo size
845 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors844 MB · 100%
At a glance
- Library
- peft
- License
- other
- Access
- Public
- Created
- Sep 26, 2024
- Updated
- Sep 28, 2024
- SHA
- 8649ab28
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
llama-7b_alpaca_l0.0002_64
This model is a fine-tuned version of huggyllama/llama-7b on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.8634
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 1
- eval_batch_size: 1
- seed: 0
- gradient_accumulation_steps: 16
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- lr_scheduler_warmup_ratio: 0.03
- training_steps: 0
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.6644 | 0.0003 | 1 | 2.9705 |
| 2.6972 | 0.0587 | 187 | 1.9034 |
| 1.8541 | 0.1174 | 374 | 1.8795 |
| 1.2478 | 0.1761 | 561 | 1.8640 |
| 1.8178 | 0.2348 | 748 | 1.8499 |
| 2.5147 | 0.2935 | 935 | 1.8168 |
| 1.4811 | 0.3522 | 1122 | 1.8192 |
| 1.5617 | 0.4108 | 1309 | 1.8117 |
| 2.6811 | 0.4695 | 1496 | 1.8032 |
| 1.9665 | 0.5282 | 1683 | 1.7964 |
| 1.5309 | 0.5869 | 1870 | 1.7978 |
| 1.5708 | 0.6456 | 2057 | 1.8058 |
| 2.2185 | 0.7043 | 2244 | 1.7870 |
| 2.6575 | 0.7630 | 2431 | 1.7815 |
| 1.6179 | 0.8217 | 2618 | 1.7880 |
| 1.4937 | 0.8804 | 2805 | 1.7864 |
| 2.2623 | 0.9391 | 2992 | 1.7782 |
| 2.4709 | 0.9978 | 3179 | 1.7754 |
| 1.9605 | 1.0565 | 3366 | 1.7927 |
| 1.5008 | 1.1151 | 3553 | 1.7958 |
| 1.4912 | 1.1738 | 3740 | 1.8099 |
| 1.9176 | 1.2325 | 3927 | 1.8007 |
| 1.5569 | 1.2912 | 4114 | 1.7962 |
| 1.3717 | 1.3499 | 4301 | 1.8071 |
| 1.5241 | 1.4086 | 4488 | 1.8020 |
| 2.1042 | 1.4673 | 4675 | 1.7964 |
| 1.6643 | 1.5260 | 4862 | 1.7947 |
| 1.3815 | 1.5847 | 5049 | 1.7994 |
| 2.5619 | 1.6434 | 5236 | 1.7989 |
| 1.7651 | 1.7021 | 5423 | 1.7948 |
| 1.4931 | 1.7608 | 5610 | 1.7908 |
| 1.5089 | 1.8195 | 5797 | 1.7957 |
| 1.768 | 1.8781 | 5984 | 1.7989 |
| 1.769 | 1.9368 | 6171 | 1.7915 |
| 1.5345 | 1.9955 | 6358 | 1.7887 |
| 1.2575 | 2.0542 | 6545 | 1.8514 |
| 1.1761 | 2.1129 | 6732 | 1.8809 |
| 1.4524 | 2.1716 | 6919 | 1.8932 |
| 1.5745 | 2.2303 | 7106 | 1.8655 |
| 1.1251 | 2.2890 | 7293 | 1.8609 |
| 1.2381 | 2.3477 | 7480 | 1.8901 |
| 1.7963 | 2.4064 | 7667 | 1.8743 |
| 1.4293 | 2.4651 | 7854 | 1.8580 |
| 1.3278 | 2.5238 | 8041 | 1.8687 |
| 1.2364 | 2.5824 | 8228 | 1.9165 |
| 1.5239 | 2.6411 | 8415 | 1.8834 |
| 1.3108 | 2.6998 | 8602 | 1.8617 |
| 1.2084 | 2.7585 | 8789 | 1.8702 |
| 1.3279 | 2.8172 | 8976 | 1.8786 |
| 1.7506 | 2.8759 | 9163 | 1.8734 |
| 1.4208 | 2.9346 | 9350 | 1.8601 |
| 1.2449 | 2.9933 | 9537 | 1.8668 |
Framework versions
- PEFT 0.12.1.dev0
- Transformers 4.45.0.dev0
- Pytorch 2.3.0+cu121
- Datasets 2.19.0
- Tokenizers 0.19.1
- Library
- peft
- License
- other
- Created
- Sep 26, 2024
- Updated
- Sep 28, 2024