MentaLLaMA-chat-7B-PsyCourse-fold8

chchen/MentaLLaMA-chat-7B-PsyCourse-fold8

MentaLLaMA-chat-7B-PsyCourse-fold8 is a machine learning model from chchen. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as mit.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

58% of all-time downloads

All-time downloads

Public

Repo size

5.3 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors80 MB · 95%

At a glance

Library: peft
License: mit
Access: Public
Created: Apr 28, 2025
Updated: Apr 28, 2025
SHA: c1c68240

Base models

klyang/MentaLLaMA-chat-7B-hf

Model card

From the Hugging Face model README

MentaLLaMA-chat-7B-PsyCourse-fold8

This model is a fine-tuned version of klyang/MentaLLaMA-chat-7B-hf on the course-train-fold8 dataset. It achieves the following results on the evaluation set:

Loss: 0.0313

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 1
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 16
total_train_batch_size: 16
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss
0.8816	0.0758	50	0.6521
0.1876	0.1516	100	0.1201
0.0847	0.2275	150	0.0709
0.0547	0.3033	200	0.0595
0.05	0.3791	250	0.0510
0.0566	0.4549	300	0.0494
0.057	0.5308	350	0.0461
0.0323	0.6066	400	0.0422
0.0331	0.6824	450	0.0393
0.0339	0.7582	500	0.0405
0.0432	0.8340	550	0.0383
0.0332	0.9099	600	0.0361
0.0458	0.9857	650	0.0381
0.0281	1.0615	700	0.0368
0.0222	1.1373	750	0.0382
0.0272	1.2132	800	0.0346
0.0303	1.2890	850	0.0352
0.0318	1.3648	900	0.0358
0.0233	1.4406	950	0.0353
0.0263	1.5164	1000	0.0349
0.0381	1.5923	1050	0.0354
0.0267	1.6681	1100	0.0319
0.0335	1.7439	1150	0.0320
0.0228	1.8197	1200	0.0320
0.0311	1.8956	1250	0.0313
0.0228	1.9714	1300	0.0314
0.0157	2.0472	1350	0.0321
0.0129	2.1230	1400	0.0316
0.0208	2.1988	1450	0.0330
0.0191	2.2747	1500	0.0325
0.0189	2.3505	1550	0.0325
0.0161	2.4263	1600	0.0325
0.0164	2.5021	1650	0.0356
0.0144	2.5780	1700	0.0338
0.0211	2.6538	1750	0.0328
0.0195	2.7296	1800	0.0322
0.0148	2.8054	1850	0.0338
0.0249	2.8812	1900	0.0327
0.0152	2.9571	1950	0.0320
0.0136	3.0329	2000	0.0329
0.009	3.1087	2050	0.0341
0.0089	3.1845	2100	0.0367
0.0127	3.2604	2150	0.0364
0.0119	3.3362	2200	0.0358
0.0118	3.4120	2250	0.0358
0.0084	3.4878	2300	0.0371
0.0137	3.5636	2350	0.0365
0.0093	3.6395	2400	0.0364
0.0095	3.7153	2450	0.0359
0.0102	3.7911	2500	0.0381
0.0137	3.8669	2550	0.0384
0.01	3.9428	2600	0.0376
0.0058	4.0186	2650	0.0389
0.0071	4.0944	2700	0.0407
0.0051	4.1702	2750	0.0414
0.0077	4.2460	2800	0.0419
0.0039	4.3219	2850	0.0424
0.002	4.3977	2900	0.0429
0.0036	4.4735	2950	0.0433
0.0083	4.5493	3000	0.0439
0.0049	4.6252	3050	0.0439
0.0033	4.7010	3100	0.0443
0.0034	4.7768	3150	0.0443
0.0038	4.8526	3200	0.0442
0.0029	4.9284	3250	0.0443

Framework versions

PEFT 0.12.0
Transformers 4.46.1
Pytorch 2.5.1+cu124
Datasets 3.1.0
Tokenizers 0.20.3

Library: peft
License: mit
Created: Apr 28, 2025
Updated: Apr 28, 2025