Downloads · 30 days
3
4% of all-time downloads
llama3-mentalhealth-lora
Omarmoh22/llama3-mentalhealth-lora
llama3-mentalhealth-lora is a machine learning model from Omarmoh22. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as llama3.2.
should probably proofread and complete it, then remove this comment. --
All-time downloads
67
Public
Repo size
145 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors36.7 MB · 68%
At a glance
- Library
- peft
- License
- llama3.2
- Access
- Public
- Created
- Mar 18, 2025
- Updated
- Mar 23, 2025
- SHA
- 50ddb7d3
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
llama3-mentalhealth-lora
This model is a fine-tuned version of meta-llama/Llama-3.2-3B-Instruct on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.0961
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 4
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 1
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.4389 | 0.0387 | 30 | 1.4279 |
| 1.3152 | 0.0774 | 60 | 1.3127 |
| 1.2543 | 0.1161 | 90 | 1.2378 |
| 1.2401 | 0.1547 | 120 | 1.1978 |
| 1.179 | 0.1934 | 150 | 1.1773 |
| 1.1067 | 0.2321 | 180 | 1.1628 |
| 1.1272 | 0.2708 | 210 | 1.1526 |
| 1.198 | 0.3095 | 240 | 1.1443 |
| 1.1318 | 0.3482 | 270 | 1.1369 |
| 1.1394 | 0.3868 | 300 | 1.1318 |
| 1.1094 | 0.4255 | 330 | 1.1266 |
| 1.1909 | 0.4642 | 360 | 1.1228 |
| 1.1212 | 0.5029 | 390 | 1.1182 |
| 1.0792 | 0.5416 | 420 | 1.1145 |
| 1.0998 | 0.5803 | 450 | 1.1115 |
| 1.0635 | 0.6190 | 480 | 1.1084 |
| 1.081 | 0.6576 | 510 | 1.1062 |
| 1.0922 | 0.6963 | 540 | 1.1041 |
| 1.1165 | 0.7350 | 570 | 1.1020 |
| 1.0789 | 0.7737 | 600 | 1.1001 |
| 1.1166 | 0.8124 | 630 | 1.0989 |
| 1.1023 | 0.8511 | 660 | 1.0980 |
| 1.0757 | 0.8897 | 690 | 1.0971 |
| 1.0887 | 0.9284 | 720 | 1.0964 |
| 1.0861 | 0.9671 | 750 | 1.0961 |
Framework versions
- PEFT 0.14.0
- Transformers 4.48.3
- Pytorch 2.6.0+cu124
- Datasets 3.4.1
- Tokenizers 0.21.1
- Library
- peft
- License
- llama3.2
- Created
- Mar 18, 2025
- Updated
- Mar 23, 2025