distilbert-base-uncased-TIC1-finetuned-squad-es

edyfjm07/distilbert-base-uncased-TIC1-finetuned-squad-es

distilbert-base-uncased-TIC1-finetuned-squad-es is a question answering model from edyfjm07. Use it when the input is a question plus a passage. It is set up for transformers. The card lists the license as apache-2.0.

probably proofread and complete it, then remove this comment. --

Downloads · 30 days

1% of all-time downloads

All-time downloads

281

Public

Repo size

13.6 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.h5266 MB · 99%

At a glance

Task: Question Answering
Library: transformers
License: apache-2.0
Model type: distilbert
Access: Public
Created: Jun 19, 2024
Updated: Jun 19, 2024
SHA: dba01cb3

Try a prompt

Base models

Model card

From the Hugging Face model README

edyfjm07/distilbert-base-uncased-TIC1-finetuned-squad-es

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

Train Loss: 0.0173
Train End Logits Accuracy: 0.9926
Train Start Logits Accuracy: 0.9937
Validation Loss: 0.6833
Validation End Logits Accuracy: 0.8809
Validation Start Logits Accuracy: 0.8746
Epoch: 50

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 6069, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
training_precision: float32

Training results

Train Loss	Train End Logits Accuracy	Train Start Logits Accuracy	Validation Loss	Validation End Logits Accuracy	Validation Start Logits Accuracy	Epoch
2.9391	0.3582	0.3183	1.5236	0.5611	0.5862	0
1.1532	0.6292	0.6586	0.8407	0.7085	0.7179	1
0.7185	0.7521	0.7563	0.7432	0.7962	0.7555	2
0.6133	0.7763	0.7784	0.6925	0.7712	0.7524	3
0.4777	0.8288	0.8267	0.6963	0.7524	0.7962	4
0.4441	0.8298	0.8403	0.6422	0.8182	0.7806	5
0.3896	0.8519	0.8645	0.6378	0.7900	0.8056	6
0.3642	0.8508	0.8771	0.6286	0.8088	0.7994	7
0.3068	0.8960	0.8887	0.5387	0.8433	0.8558	8
0.2755	0.8845	0.8845	0.6049	0.8245	0.8307	9
0.2711	0.9023	0.9023	0.5653	0.8527	0.8370	10
0.2260	0.9065	0.9202	0.6267	0.8589	0.8150	11
0.2016	0.9086	0.9286	0.6035	0.8777	0.8401	12
0.2044	0.9107	0.9296	0.5305	0.8840	0.8683	13
0.1923	0.9275	0.9296	0.5440	0.8871	0.8621	14
0.1448	0.9307	0.9496	0.5563	0.8934	0.8464	15
0.1465	0.9359	0.9454	0.5626	0.8809	0.8589	16
0.1323	0.9464	0.9517	0.6286	0.8401	0.8495	17
0.1291	0.9506	0.9506	0.5277	0.8746	0.8621	18
0.1156	0.9590	0.9559	0.5341	0.8777	0.8558	19
0.0839	0.9643	0.9748	0.5753	0.8903	0.8527	20
0.1007	0.9538	0.9653	0.5299	0.8746	0.8558	21
0.0901	0.9664	0.9664	0.6034	0.8558	0.8464	22
0.0791	0.9716	0.9779	0.6137	0.8777	0.8495	23
0.0782	0.9653	0.9748	0.6260	0.8809	0.8589	24
0.0747	0.9748	0.9748	0.5973	0.8903	0.8527	25
0.0685	0.9653	0.9821	0.6007	0.8809	0.8777	26
0.0688	0.9685	0.9737	0.5546	0.8903	0.8495	27
0.0513	0.9811	0.9842	0.5925	0.8997	0.8495	28
0.0518	0.9769	0.9863	0.6222	0.8777	0.8746	29
0.0451	0.9748	0.9916	0.6302	0.8777	0.8746	30
0.0424	0.9842	0.9811	0.6389	0.8871	0.8652	31
0.0392	0.9800	0.9853	0.6361	0.8809	0.8715	32
0.0382	0.9842	0.9895	0.6253	0.8840	0.8715	33
0.0405	0.9800	0.9905	0.6734	0.8715	0.8777	34
0.0405	0.9769	0.9905	0.6104	0.8903	0.8652	35
0.0364	0.9790	0.9926	0.6584	0.8809	0.8715	36
0.0272	0.9842	0.9947	0.6439	0.8871	0.8715	37
0.0240	0.9916	0.9937	0.6390	0.8934	0.8746	38
0.0211	0.9884	0.9958	0.6597	0.8871	0.8683	39
0.0277	0.9916	0.9926	0.6561	0.8809	0.8683	40
0.0307	0.9884	0.9874	0.6669	0.8809	0.8652	41
0.0186	0.9947	0.9947	0.6526	0.8871	0.8652	42
0.0178	0.9905	0.9958	0.6681	0.8840	0.8621	43
0.0195	0.9905	0.9926	0.6780	0.8903	0.8683	44
0.0197	0.9937	0.9916	0.7142	0.8777	0.8558	45
0.0176	0.9947	0.9926	0.6914	0.8809	0.8715	46
0.0188	0.9947	0.9916	0.6901	0.8809	0.8652	47
0.0150	0.9958	0.9926	0.6845	0.8809	0.8715	48
0.0170	0.9937	0.9905	0.6826	0.8840	0.8715	49
0.0173	0.9926	0.9937	0.6833	0.8809	0.8746	50

Framework versions

Transformers 4.41.2
TensorFlow 2.15.0
Datasets 2.20.0
Tokenizers 0.19.1

Task: Question Answering
Library: transformers
Type: distilbert
License: apache-2.0
Created: Jun 19, 2024
Updated: Jun 19, 2024