PELM-JointGPT

GItaf/PELM-JointGPT

PELM-JointGPT is a text generation model from GItaf. Use it when you need the model to write or continue text. It is set up for transformers.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

5% of all-time downloads

All-time downloads

613

Public

Parameters

137M

6.1 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin510 MB · 50%

Parameter types

How the weights are stored.

F32124M · 91%

Try a prompt

Model card

From the Hugging Face model README

PELM-JointGPT2

This model is based on PELM framework and initialised from genGPT-2, then fine-tuned on the MBTI dataset. It achieves the following results on the evaluation set:

Loss: 4.3556
Cls loss: 1.5778
Lm loss: 3.9609
Cls Accuracy: 0.6202
Cls F1: 0.6126
Cls Precision: 0.6216
Cls Recall: 0.6202
Perplexity: 52.50

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 2
eval_batch_size: 2
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Cls loss	Lm loss	Cls Accuracy	Cls F1	Cls Precision	Cls Recall	Perplexity
4.2735	1.0	3470	4.3562	1.5844	3.9598	0.5833	0.5708	0.5928	0.5833	52.45
4.0754	2.0	6940	4.3295	1.4806	3.9590	0.6196	0.6113	0.6332	0.6196	52.41
3.985	3.0	10410	4.3556	1.5778	3.9609	0.6202	0.6126	0.6216	0.6202	52.50

Framework versions

Transformers 4.21.2
Pytorch 1.12.1
Datasets 2.4.0
Tokenizers 0.12.1

Task: Text Generation
Library: transformers
Type: gpt2
Created: Nov 9, 2022
Updated: Jul 14, 2023