gpt2_sm_cv_summarized_4

wiorz/gpt2_sm_cv_summarized_4

gpt2_sm_cv_summarized_4 is a text classification model from wiorz. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as mit.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

33% of all-time downloads

All-time downloads

Public

Parameters

137M

1 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin510 MB · 50%

Parameter types

How the weights are stored.

F32124M · 91%

Try a prompt

Model card

From the Hugging Face model README

gpt2_sm_cv_summarized_4

This model is a fine-tuned version of gpt2 on the None dataset. It achieves the following results on the evaluation set:

Loss: 2.4906
Accuracy: 0.743
Precision: 0.3038
Recall: 0.2462
F1: 0.2720
D-index: 1.4383

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 8000
num_epochs: 20
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1	D-index
No log	1.0	250	1.0827	0.673	0.2155	0.2564	0.2342	1.3419
2.5863	2.0	500	0.6038	0.761	0.2381	0.1026	0.1434	1.4124
2.5863	3.0	750	0.5238	0.794	0.3333	0.0564	0.0965	1.4421
0.5205	4.0	1000	0.5206	0.798	0.3158	0.0308	0.0561	1.4384
0.5205	5.0	1250	0.5068	0.802	0.4286	0.0462	0.0833	1.4495
0.4619	6.0	1500	0.5153	0.795	0.4074	0.1128	0.1767	1.4637
0.4619	7.0	1750	0.5246	0.795	0.3611	0.0667	0.1126	1.4472
0.3988	8.0	2000	0.6671	0.797	0.3333	0.0410	0.0731	1.4407
0.3988	9.0	2250	0.6091	0.763	0.35	0.2513	0.2925	1.4680
0.3128	10.0	2500	0.7342	0.759	0.3284	0.2256	0.2675	1.4535
0.3128	11.0	2750	0.8236	0.741	0.3049	0.2564	0.2786	1.4391
0.2198	12.0	3000	0.9349	0.742	0.3297	0.3128	0.3211	1.4601
0.2198	13.0	3250	1.1979	0.764	0.3481	0.2410	0.2848	1.4658
0.1522	14.0	3500	1.3995	0.758	0.3464	0.2718	0.3046	1.4682
0.1522	15.0	3750	2.3304	0.779	0.3143	0.1128	0.1660	1.4414
0.1137	16.0	4000	2.0930	0.762	0.2991	0.1641	0.2119	1.4359
0.1137	17.0	4250	2.6869	0.787	0.3714	0.1333	0.1962	1.4598
0.0904	18.0	4500	2.2347	0.678	0.2818	0.4205	0.3374	1.4071
0.0904	19.0	4750	2.4580	0.752	0.3221	0.2462	0.2791	1.4509
0.0737	20.0	5000	2.4906	0.743	0.3038	0.2462	0.2720	1.4383

Framework versions

Transformers 4.28.0
Pytorch 2.0.1+cu118
Datasets 2.12.0
Tokenizers 0.13.3

Task: Text Classification
Library: transformers
Type: gpt2
License: mit
Created: Jun 8, 2023
Updated: Feb 3, 2025