gpt2-xl_ft_mult_5k

beston91/gpt2-xl_ft_mult_5k

gpt2-xl_ft_mult_5k is a text generation model from beston91. Use it when you need the model to write or continue text. It is set up for transformers.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

4% of all-time downloads

All-time downloads

395

Public

Repo size

18.8 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin6.3 GB · 100%

At a glance

Task: Text Generation
Library: transformers
Model type: gpt2
Access: Public
Created: Mar 19, 2022
Updated: Mar 20, 2022
SHA: 4f2b5d5e

Try a prompt

Model card

From the Hugging Face model README

gpt2-xl_ft_mult_5k

This model is a fine-tuned version of gpt2-xl on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.6758

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
gradient_accumulation_steps: 32
total_train_batch_size: 128
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 100.0
num_epochs: 4
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.99	27	6.3035
No log	1.99	54	1.2709
No log	2.99	81	0.7482
No log	3.99	108	0.6758

Framework versions

Transformers 4.17.0
Pytorch 1.10.0+cu111
Datasets 2.0.0
Tokenizers 0.11.6

Perplexity

Score: 21.267963409423828

Dataset Size

Size: 5000

Task: Text Generation
Library: transformers
Type: gpt2
Created: Mar 19, 2022
Updated: Mar 20, 2022