Downloads · 30 days
11
3% of all-time downloads
deberta-small
lucadiliello/deberta-small
deberta-small is a fill-mask model from lucadiliello. Use it when you need the model to fill a missing word. It is set up for transformers.
A small version of DeBERTa trained on the clean version of google C4 dataset. For more info about the size of the model, see config.json.
All-time downloads
395
Public
Repo size
330 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin165 MB · 98%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- Model type
- deberta
- Access
- Public
- Created
- Feb 27, 2023
- Updated
- Feb 27, 2023
- SHA
- fbf513cb
Try a prompt
Datasets
Model card
From the Hugging Face model README
A small version of DeBERTa trained on the clean version of google C4 dataset. For more info about the size of the model, see config.json.
The model has been trained for 100K steps with a batch size of 2048 and a sequence length of 512, for a total of 104B tokens.
The vocabulary and the tokenizer are the same as microsoft/deberta-base.
- Task
- Fill-Mask
- Library
- transformers
- Type
- deberta
- Languages
- en
- Created
- Feb 27, 2023
- Updated
- Feb 27, 2023