MentaLLaMA-chat-7B-PsyCourse-fold5

chchen/MentaLLaMA-chat-7B-PsyCourse-fold5

MentaLLaMA-chat-7B-PsyCourse-fold5 is a machine learning model from chchen. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as mit.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

79% of all-time downloads

All-time downloads

Public

Repo size

5.3 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors80 MB · 95%

At a glance

Library: peft
License: mit
Access: Public
Created: Apr 29, 2025
Updated: Apr 29, 2025
SHA: 72d84066

Base models

klyang/MentaLLaMA-chat-7B-hf

Model card

From the Hugging Face model README

MentaLLaMA-chat-7B-PsyCourse-fold5

This model is a fine-tuned version of klyang/MentaLLaMA-chat-7B-hf on the course-train-fold5 dataset. It achieves the following results on the evaluation set:

Loss: 0.0295

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 1
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 16
total_train_batch_size: 16
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss
0.8836	0.0758	50	0.6510
0.1276	0.1517	100	0.1150
0.0848	0.2275	150	0.0731
0.0545	0.3033	200	0.0569
0.0542	0.3791	250	0.0499
0.0466	0.4550	300	0.0510
0.0517	0.5308	350	0.0468
0.058	0.6066	400	0.0456
0.0521	0.6825	450	0.0405
0.0317	0.7583	500	0.0382
0.0281	0.8341	550	0.0390
0.0388	0.9100	600	0.0388
0.0459	0.9858	650	0.0355
0.0277	1.0616	700	0.0368
0.0342	1.1374	750	0.0369
0.0323	1.2133	800	0.0337
0.0257	1.2891	850	0.0351
0.0218	1.3649	900	0.0346
0.0266	1.4408	950	0.0377
0.0344	1.5166	1000	0.0322
0.0244	1.5924	1050	0.0315
0.0227	1.6682	1100	0.0332
0.0243	1.7441	1150	0.0318
0.03	1.8199	1200	0.0311
0.0307	1.8957	1250	0.0295
0.0344	1.9716	1300	0.0305
0.0214	2.0474	1350	0.0307
0.0178	2.1232	1400	0.0320
0.0167	2.1991	1450	0.0321
0.0115	2.2749	1500	0.0325
0.0192	2.3507	1550	0.0318
0.0233	2.4265	1600	0.0327
0.0108	2.5024	1650	0.0340
0.0256	2.5782	1700	0.0315
0.019	2.6540	1750	0.0300
0.0205	2.7299	1800	0.0302
0.0197	2.8057	1850	0.0307
0.0161	2.8815	1900	0.0303
0.0235	2.9573	1950	0.0302
0.01	3.0332	2000	0.0301
0.0073	3.1090	2050	0.0325
0.0099	3.1848	2100	0.0337
0.0085	3.2607	2150	0.0337
0.0076	3.3365	2200	0.0354
0.0077	3.4123	2250	0.0341
0.0107	3.4882	2300	0.0338
0.006	3.5640	2350	0.0338
0.0127	3.6398	2400	0.0336
0.0099	3.7156	2450	0.0338
0.014	3.7915	2500	0.0337
0.0129	3.8673	2550	0.0339
0.0118	3.9431	2600	0.0350
0.0073	4.0190	2650	0.0346
0.0048	4.0948	2700	0.0357
0.0059	4.1706	2750	0.0373
0.0053	4.2464	2800	0.0373
0.0045	4.3223	2850	0.0381
0.0054	4.3981	2900	0.0388
0.0085	4.4739	2950	0.0385
0.0066	4.5498	3000	0.0384
0.0051	4.6256	3050	0.0386
0.0052	4.7014	3100	0.0388
0.0065	4.7773	3150	0.0389
0.0036	4.8531	3200	0.0391
0.0039	4.9289	3250	0.0391

Framework versions

PEFT 0.12.0
Transformers 4.46.1
Pytorch 2.5.1+cu124
Datasets 3.1.0
Tokenizers 0.20.3

Library: peft
License: mit
Created: Apr 29, 2025
Updated: Apr 29, 2025