cervantes-gpt

ser-mei/cervantes-gpt

cervantes-gpt is a text generation model from ser-mei. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

2% of all-time downloads

All-time downloads

477

Public

Repo size

1 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin510 MB · 99%

At a glance

Task: Text Generation
Library: transformers
License: mit
Model type: gpt2
Access: Public
Created: Dec 13, 2022
Updated: Dec 13, 2022
SHA: 007b42e1

Try a prompt

Model card

From the Hugging Face model README

cervantes-gpt

This model is a fine-tuned version of DeepESP/gpt2-spanish on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 8.1302

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 32
eval_batch_size: 32
seed: 42
gradient_accumulation_steps: 16
total_train_batch_size: 512
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 500
num_epochs: 70
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
10.6864	0.96	13	9.4380
9.6293	1.96	26	9.0791
9.2039	2.96	39	8.5999
8.5709	3.96	52	7.9434
7.8331	4.96	65	7.2929
7.1731	5.96	78	6.7935
6.681	6.96	91	6.4989
6.359	7.96	104	6.3480
6.1194	8.96	117	6.1738
5.8887	9.96	130	6.0409
5.6722	10.96	143	5.9433
5.4738	11.96	156	5.8746
5.2853	12.96	169	5.7898
5.1082	13.96	182	5.7821
4.9458	14.96	195	5.7489
4.7782	15.96	208	5.7815
4.613	16.96	221	5.7930
4.4529	17.96	234	5.8027
4.2796	18.96	247	5.8341
4.0998	19.96	260	5.8972
3.9184	20.96	273	6.0337
3.7264	21.96	286	6.0392
3.5419	22.96	299	6.1160
3.3477	23.96	312	6.2168
3.1492	24.96	325	6.2471
2.9641	25.96	338	6.3488
2.7695	26.96	351	6.4372
2.5882	27.96	364	6.4921
2.4007	28.96	377	6.6257
2.2178	29.96	390	6.6335
2.0489	30.96	403	6.7425
1.8779	31.96	416	6.7861
1.7209	32.96	429	6.8796
1.5707	33.96	442	6.9420
1.3984	34.96	455	6.9857
1.2653	35.96	468	7.0169
1.1368	36.96	481	7.0835
1.01	37.96	494	7.1329
0.8959	38.96	507	7.2498
0.792	39.96	520	7.2971
0.6844	40.96	533	7.2841
0.6028	41.96	546	7.3295
0.5216	42.96	559	7.3776
0.467	43.96	572	7.4190
0.417	44.96	585	7.5201
0.3785	45.96	598	7.5042
0.3456	46.96	611	7.5822
0.3164	47.96	624	7.6342
0.2882	48.96	637	7.6722
0.2674	49.96	650	7.6951
0.2471	50.96	663	7.7717
0.2287	51.96	676	7.8266
0.2116	52.96	689	7.8124
0.195	53.96	702	7.8595
0.1784	54.96	715	7.8968
0.1633	55.96	728	7.9242
0.1491	56.96	741	7.9956
0.1412	57.96	754	8.0052
0.1338	58.96	767	8.0319
0.1284	59.96	780	8.0596
0.1229	60.96	793	8.0776
0.1193	61.96	806	8.0791
0.117	62.96	819	8.0912
0.1142	63.96	832	8.1174
0.1129	64.96	845	8.1141
0.1114	65.96	858	8.1197
0.11	66.96	871	8.1275
0.1102	67.96	884	8.1302
0.1093	68.96	897	8.1303
0.1046	69.96	910	8.1302

Framework versions

Transformers 4.24.0
Pytorch 1.13.0+rocm5.2
Datasets 2.6.1
Tokenizers 0.13.2

Task: Text Generation
Library: transformers
Type: gpt2
License: mit
Created: Dec 13, 2022
Updated: Dec 13, 2022