Downloads · 30 days
36
4% of all-time downloads
bert-base-uncased-sparse-1_2
Intel/bert-base-uncased-sparse-1_2
bert-base-uncased-sparse-1_2 is a machine learning model from Intel. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
Pretrained model pruned to 1:2 structured sparsity. The model is a pruned version of the BERT base model.
All-time downloads
821
Public
Repo size
881 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin441 MB · 100%
At a glance
- Library
- transformers
- Model type
- bert
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Jun 24, 2021
- SHA
- eee4f164
Model card
From the Hugging Face model README
Sparse BERT base model (uncased)
Pretrained model pruned to 1:2 structured sparsity. The model is a pruned version of the BERT base model.
Intended Use
The model can be used for fine-tuning to downstream tasks with sparsity already embeded to the model. To keep the sparsity a mask should be added to each sparse weight blocking the optimizer from updating the zeros.
Evaluation Results
We get the following results on the tasks development set, all results are mean of 5 different seeded models:
| Task | MNLI-m (Acc) | MNLI-mm (Acc) | QQP (Acc/F1) | QNLI (Acc) | SST-2 (Acc) | STS-B (Pears/Spear) | SQuADv1.1 (Acc/F1) |
|---|---|---|---|---|---|---|---|
| 83.3 | 83.9 | 90.8/87.6 | 90.4 | 91.3 | 88.8/88.3 | 80.5/88.2 |
- Library
- transformers
- Type
- bert
- Languages
- en
- Created
- Mar 2, 2022
- Updated
- Jun 24, 2021