long-t5-tglobal-base-boardpapers-4096

RMWeerasinghe/long-t5-tglobal-base-boardpapers-4096

long-t5-tglobal-base-boardpapers-4096 is a summarization model from RMWeerasinghe. Use it when you need a shorter version of a longer text. It is set up for transformers. The card lists the license as apache-2.0.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

4% of all-time downloads

All-time downloads

312

Public

Parameters

248M

2 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors990 MB · 100%

At a glance

Task: Summarization
Library: transformers
License: apache-2.0
Model type: longt5
Access: Public
Created: Feb 14, 2024
Updated: Feb 15, 2024
SHA: bca055cd

Try a prompt

Base models

RMWeerasinghe/long-t5-tglobal-base-finetuned-govReport-4096

Model card

From the Hugging Face model README

long-t5-tglobal-base-boardpapers-4096

This model is a fine-tuned version of RMWeerasinghe/long-t5-tglobal-base-finetuned-govReport-4096 on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.5356
Rouge1: 0.0844
Rouge2: 0.0543
Rougel: 0.0716
Rougelsum: 0.0842

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 4e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
No log	0.67	1	0.6583	0.0647	0.03	0.0504	0.0595
No log	2.0	3	0.6232	0.067	0.036	0.0527	0.0643
No log	2.67	4	0.6134	0.067	0.036	0.0527	0.0643
No log	4.0	6	0.5971	0.0742	0.0426	0.0654	0.0735
No log	4.67	7	0.5897	0.0765	0.0462	0.0654	0.0762
No log	6.0	9	0.5777	0.0803	0.0486	0.0665	0.0802
No log	6.67	10	0.5729	0.0813	0.0498	0.0677	0.0801
No log	8.0	12	0.5652	0.0813	0.0498	0.0677	0.0801
No log	8.67	13	0.5622	0.0823	0.0544	0.0685	0.0811
No log	10.0	15	0.5575	0.0823	0.0544	0.0685	0.0811
No log	10.67	16	0.5559	0.0823	0.0544	0.0685	0.0811
No log	12.0	18	0.5528	0.0823	0.0544	0.0685	0.0811
No log	12.67	19	0.5513	0.0823	0.0544	0.0685	0.0811
0.7235	14.0	21	0.5488	0.0823	0.0544	0.0685	0.0811
0.7235	14.67	22	0.5476	0.0811	0.0544	0.0674	0.0794
0.7235	16.0	24	0.5451	0.086	0.0574	0.074	0.0841
0.7235	16.67	25	0.5438	0.086	0.0574	0.074	0.0841
0.7235	18.0	27	0.5420	0.086	0.0574	0.074	0.0841
0.7235	18.67	28	0.5412	0.086	0.0574	0.074	0.0841
0.7235	20.0	30	0.5397	0.086	0.0574	0.074	0.0841
0.7235	20.67	31	0.5390	0.086	0.0574	0.074	0.0841
0.7235	22.0	33	0.5377	0.0844	0.0543	0.0716	0.0842
0.7235	22.67	34	0.5372	0.0844	0.0543	0.0716	0.0842
0.7235	24.0	36	0.5363	0.0844	0.0543	0.0716	0.0842
0.7235	24.67	37	0.5360	0.0844	0.0543	0.0716	0.0842
0.7235	26.0	39	0.5357	0.0844	0.0543	0.0716	0.0842
0.6478	26.67	40	0.5356	0.0844	0.0543	0.0716	0.0842

Framework versions

Transformers 4.37.0
Pytorch 2.1.2
Datasets 2.17.0
Tokenizers 0.15.1

Task: Summarization
Library: transformers
Type: longt5
License: apache-2.0
Created: Feb 14, 2024
Updated: Feb 15, 2024