Downloads · 30 days
6
2% of all-time downloads
Distilled-RoBERTa
Matthijs0/Distilled-RoBERTa
Distilled-RoBERTa is a question answering model from Matthijs0. Use it when the input is a question plus a passage. It is set up for transformers. The card lists the license as mit.
The DistilBERT model is a RoBERTa model, which is trained on the SQuAD 2.0 training set, fine-tuned on the NewsQA dataset.
All-time downloads
299
Public
Parameters
124M
496 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors496 MB · 99%
At a glance
- Task
- Question Answering
- Library
- transformers
- License
- mit
- Model type
- roberta
- Access
- Public
- Created
- Apr 8, 2024
- Updated
- Apr 8, 2024
- SHA
- 6fc12dfa
Try a prompt
Model card
From the Hugging Face model README
Distilled-RoBERTa
The DistilBERT model is a RoBERTa model, which is trained on the SQuAD 2.0 training set, fine-tuned on the NewsQA dataset.
Hyperparameters
batch_size = 16
n_epochs = 3
max_seq_len = 512
learning_rate = 2e-5
optimizer=AdamW
lr_schedule = LinearWarmup
weight_decay=0.01
embeds_dropout_prob = 0.1
- Task
- Question Answering
- Library
- transformers
- Type
- roberta
- License
- mit
- Created
- Apr 8, 2024
- Updated
- Apr 8, 2024