LLM_Teached_Pegasus_100k

GlycerinLOL/LLM_Teached_Pegasus_100k

LLM_Teached_Pegasus_100k is a machine learning model from GlycerinLOL. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

3% of all-time downloads

All-time downloads

196

Public

Parameters

570M

59.3 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors2.3 GB · 100%

At a glance

Library: transformers
Model type: pegasus
Access: Public
Created: Mar 24, 2024
Updated: Mar 25, 2024
SHA: 164b1848

Base models

google/pegasus-xsum

Model card

From the Hugging Face model README

LLM_Teached_Pegasus_100k

This model is a fine-tuned version of google/pegasus-xsum on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.5004
Rouge1: 0.4923
Rouge2: 0.2429
Rougel: 0.4134
Rougelsum: 0.4134
Gen Len: 25.1335
Precision: 0.9143
Recall: 0.9124
F1: 0.9132

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 32
eval_batch_size: 16
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 128
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 16
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	F1	Gen Len	Validation Loss	Precision	Recall	Rouge1	Rouge2	Rougel	Rougelsum
2.1501	1.0	781	0.9072	25.4655	1.7062	0.9082	0.9065	0.4566	0.209	0.3745	0.3744
1.7722	2.0	1562	0.9097	25.4298	1.6314	0.9107	0.909	0.4712	0.2226	0.3906	0.3904
1.7218	3.0	2343	0.9106	25.6569	1.5948	0.9112	0.9103	0.4776	0.2284	0.3965	0.3963
1.6668	4.0	3125	0.9112	25.3451	1.5708	0.9122	0.9107	0.481	0.2316	0.4002	0.4
1.6437	5.0	3906	0.9118	25.482	1.5565	0.9127	0.9113	0.4844	0.2346	0.4034	0.4031
1.6186	6.0	4687	0.912	25.4191	1.5476	0.9129	0.9115	0.4852	0.236	0.4047	0.4044
1.607	7.0	5468	0.9122	25.4949	1.5426	0.9129	0.9118	0.486	0.2367	0.4052	0.405
1.5972	8.0	6248	1.5380	0.4872	0.2387	0.407	0.4071	25.3836	0.9131	0.9118	0.9123
1.5836	9.0	7029	1.5273	0.4891	0.2399	0.4088	0.4089	25.4995	0.9133	0.9122	0.9126
1.5667	10.0	7810	1.5196	0.4906	0.2416	0.411	0.4112	25.3867	0.9135	0.9123	0.9127
1.5521	11.0	8592	1.5124	0.4899	0.2406	0.4102	0.4103	25.2191	0.9137	0.912	0.9127
1.5413	12.0	9373	1.5083	0.4914	0.2416	0.4118	0.412	25.3491	0.9137	0.9123	0.9128
1.5291	13.0	10154	1.5044	0.4913	0.2419	0.4118	0.4119	25.2082	0.914	0.9123	0.913
1.527	14.0	10935	1.5026	0.4917	0.2426	0.4126	0.4128	25.1069	0.9141	0.9123	0.913
1.5203	15.0	11717	1.5006	0.4921	0.243	0.4135	0.4136	25.1062	0.9143	0.9123	0.9131
1.5126	16.0	12496	1.5004	0.4923	0.2429	0.4134	0.4134	25.1335	0.9143	0.9124	0.9132

Framework versions

Transformers 4.36.0
Pytorch 2.0.1+cu117
Datasets 2.14.5
Tokenizers 0.15.0

Library: transformers
Type: pegasus
Created: Mar 24, 2024
Updated: Mar 25, 2024