t5-small-billsum_model

alaahussein/t5-small-billsum_model

t5-small-billsum_model is a machine learning model from alaahussein. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

4% of all-time downloads

All-time downloads

216

Public

Repo size

9.9 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin242 MB · 99%

At a glance

Library: transformers
License: apache-2.0
Model type: t5
Access: Public
Created: Apr 15, 2023
Updated: Apr 15, 2023
SHA: 89791622

Datasets

billsum

Model card

From the Hugging Face model README

t5-small-billsum_model

This model is a fine-tuned version of t5-small on the billsum dataset. It achieves the following results on the evaluation set:

Loss: 1.4109
Rouge1: 0.2444
Rouge2: 0.2013
Rougel: 0.2371
Rougelsum: 0.2372
Gen Len: 18.9994
Bleu: 0.0018

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 64
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 40
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len	Bleu
No log	1.0	296	1.8388	0.222	0.1724	0.2118	0.2119	18.9942	0.0011
2.2619	2.0	592	1.7177	0.2287	0.1797	0.2186	0.2187	18.9957	0.0013
2.2619	3.0	888	1.6596	0.2337	0.1853	0.2241	0.2242	18.9979	0.0015
1.9013	4.0	1185	1.6184	0.2359	0.1873	0.2268	0.2269	19.0	0.0016
1.9013	5.0	1481	1.5934	0.2372	0.1891	0.2285	0.2286	19.0	0.0016
1.8196	6.0	1777	1.5683	0.2378	0.1901	0.2293	0.2294	18.9985	0.0016
1.7679	7.0	2073	1.5489	0.2371	0.1901	0.2288	0.2289	19.0	0.0016
1.7679	8.0	2370	1.5335	0.2386	0.1924	0.2306	0.2306	19.0	0.0017
1.7315	9.0	2666	1.5215	0.239	0.193	0.2311	0.2312	19.0	0.0017
1.7315	10.0	2962	1.5094	0.2394	0.1938	0.2318	0.2317	19.0	0.0017
1.6994	11.0	3258	1.4996	0.2403	0.195	0.2325	0.2325	19.0	0.0017
1.6769	12.0	3555	1.4904	0.2405	0.1955	0.2328	0.2328	19.0	0.0017
1.6769	13.0	3851	1.4820	0.2409	0.1961	0.2333	0.2333	19.0	0.0017
1.659	14.0	4147	1.4740	0.2416	0.1971	0.2341	0.2341	18.9994	0.0017
1.659	15.0	4443	1.4694	0.242	0.1978	0.2345	0.2346	19.0	0.0018
1.6436	16.0	4740	1.4629	0.2425	0.1981	0.2348	0.2348	19.0	0.0018
1.6271	17.0	5036	1.4567	0.2428	0.1987	0.2352	0.2353	18.9994	0.0018
1.6271	18.0	5332	1.4519	0.243	0.1991	0.2355	0.2356	19.0	0.0018
1.6135	19.0	5628	1.4483	0.2429	0.1992	0.2354	0.2355	18.9994	0.0018
1.6135	20.0	5925	1.4425	0.2434	0.1998	0.2359	0.236	18.9994	0.0018
1.6038	21.0	6221	1.4403	0.2435	0.1999	0.2361	0.2362	18.9994	0.0018
1.5958	22.0	6517	1.4370	0.2436	0.2	0.2363	0.2364	18.9994	0.0018
1.5958	23.0	6813	1.4328	0.2439	0.2003	0.2366	0.2367	18.9994	0.0018
1.5875	24.0	7110	1.4308	0.2439	0.2003	0.2366	0.2367	18.9994	0.0018
1.5875	25.0	7406	1.4283	0.2439	0.2004	0.2366	0.2367	18.9994	0.0018
1.581	26.0	7702	1.4255	0.2438	0.2003	0.2365	0.2367	18.9994	0.0018
1.581	27.0	7998	1.4241	0.2438	0.2005	0.2365	0.2366	18.9994	0.0018
1.5734	28.0	8295	1.4212	0.244	0.2007	0.2367	0.2368	18.9994	0.0018
1.5697	29.0	8591	1.4199	0.244	0.2007	0.2367	0.2368	18.9994	0.0018
1.5697	30.0	8887	1.4173	0.244	0.2007	0.2368	0.2368	18.9994	0.0018
1.5639	31.0	9183	1.4168	0.2439	0.2007	0.2367	0.2368	18.9994	0.0018
1.5639	32.0	9480	1.4159	0.2441	0.2007	0.2367	0.2368	18.9994	0.0018
1.5608	33.0	9776	1.4143	0.2442	0.2009	0.2369	0.237	18.9994	0.0018
1.5562	34.0	10072	1.4132	0.2442	0.2009	0.2369	0.237	18.9994	0.0018
1.5562	35.0	10368	1.4123	0.2442	0.201	0.2369	0.237	18.9994	0.0018
1.5563	36.0	10665	1.4122	0.2443	0.2012	0.237	0.2371	18.9994	0.0018
1.5563	37.0	10961	1.4112	0.2443	0.2011	0.237	0.2371	18.9994	0.0018
1.5526	38.0	11257	1.4112	0.2444	0.2013	0.2371	0.2373	18.9994	0.0018
1.5525	39.0	11553	1.4110	0.2443	0.2012	0.237	0.2372	18.9994	0.0018
1.5525	39.97	11840	1.4109	0.2444	0.2013	0.2371	0.2372	18.9994	0.0018

Framework versions

Transformers 4.28.1
Pytorch 2.0.0+cu118
Datasets 2.11.0
Tokenizers 0.13.3

Library: transformers
Type: t5
License: apache-2.0
Created: Apr 15, 2023
Updated: Apr 15, 2023