mt5-kormath

madatnlp/mt5-kormath

mt5-kormath is a machine learning model from madatnlp. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.

probably proofread and complete it, then remove this comment. --

Downloads · 30 days

8% of all-time downloads

All-time downloads

211

Public

Repo size

8.6 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.h51.7 GB · 99%

At a glance

Library: transformers
License: apache-2.0
Model type: mt5
Access: Public
Created: May 10, 2022
Updated: May 11, 2022
SHA: 1041fb43

Model card

From the Hugging Face model README

madatnlp/mt5-kormath

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

Train Loss: 0.7119
Validation Loss: 1.1299
Epoch: 61

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

optimizer: {'name': 'Adam', 'learning_rate': 0.001, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
training_precision: mixed_bfloat16

Training results

Train Loss	Validation Loss	Epoch
17.9929	5.9287	0
5.4802	3.9942	1
4.1718	3.2517	2
3.5750	2.9586	3
3.1535	2.4970	4
2.8665	2.4626	5
2.6682	2.3795	6
2.5323	2.2238	7
2.4057	2.0684	8
2.3107	2.2033	9
2.2501	1.8339	10
2.1089	1.9064	11
2.0741	2.0256	12
1.9868	1.8107	13
1.9719	1.7157	14
1.8762	1.6966	15
1.8814	1.6580	16
1.8052	1.6043	17
1.7567	1.6572	18
1.7209	1.5485	19
1.7347	1.6464	20
1.6760	1.5892	21
1.6286	1.5765	22
1.6124	1.7408	23
1.5683	1.4875	24
1.5814	1.4448	25
1.5306	1.4902	26
1.5121	1.5133	27
1.4869	1.4217	28
1.4539	1.5602	29
1.4650	1.4699	30
1.4508	1.4319	31
1.3910	1.5975	32
1.3758	1.4031	33
1.3550	1.4295	34
1.3405	1.3804	35
1.3144	1.4202	36
1.3136	1.5135	37
1.2617	1.4790	38
1.2260	1.4108	39
1.2348	1.3108	40
1.2019	1.1461	41
1.1775	1.2509	42
1.1690	1.2179	43
1.1318	1.2483	44
1.1013	1.0815	45
1.0735	1.2135	46
1.0439	1.1260	47
1.0182	1.1993	48
0.9971	1.0797	49
0.9583	1.2587	50
0.9505	1.0793	51
0.9366	1.0501	52
0.9170	1.1476	53
0.8741	1.0560	54
0.8558	1.0024	55
0.8394	0.9604	56
0.8203	1.2700	57
0.7938	1.1081	58
0.7800	1.0198	59
0.7378	1.1748	60
0.7119	1.1299	61

Framework versions

Transformers 4.18.0
TensorFlow 2.8.0
Datasets 2.2.0
Tokenizers 0.12.1

Library: transformers
Type: mt5
License: apache-2.0
Created: May 10, 2022
Updated: May 11, 2022