Downloads · 30 days
0
eftnas-s2-bert-medium
IntelLabs/eftnas-s2-bert-medium
eftnas-s2-bert-medium is a machine learning model from IntelLabs. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
The super-networks fine-tuned on BERT-medium with GLUE benchmark using EFTNAS.
Access
Public
Updated Mar 19, 2024
Repo size
4 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin3.9 GB · 94%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Mar 15, 2024
- Updated
- Mar 19, 2024
- SHA
- d89dade1
Datasets
Model card
From the Hugging Face model README
EFTNAS Model Card: eftnas-s2-bert-medium
The super-networks fine-tuned on BERT-medium with GLUE benchmark using EFTNAS.
Model Details
Information
- Model name: eftnas-s2-bert-medium-[TASK]
- Base model: google/bert_uncased_L-8_H-512_A-8
- Subnetwork version: Super-network
- NNCF Configurations: eftnas_configs
Training and Evaluation
Results
Results of the optimal sub-network discoverd from the super-network:
| Model | GFLOPs | GLUE Avg. | MNLI-m | QNLI | QQP | SST-2 | CoLA | MRPC | RTE |
|---|---|---|---|---|---|---|---|---|---|
| Test Set: | |||||||||
| EFTNAS-S1 | 5.7 | 77.7 | 83.7 | 89.9 | 71.8 | 93.4 | 52.6 | 87.6 | 65.0 |
| EFTNAS-S2 | 2.2 | 75.2 | 82.0 | 87.8 | 70.6 | 91.4 | 44.5 | 86.1 | 64.0 |
Model Sources
- Repository: https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/tree/main/EFTNAS
- Paper: Searching for Efficient Language Models in First-Order Weight-Reordered Super-Networks
Citation
@inproceedings{
eftnas2024,
title={Searching for Efficient Language Models in First-Order Weight-Reordered Super-Networks},
author={J. Pablo Munoz and Yi Zheng and Nilesh Jain},
booktitle={The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation},
year={2024},
url={}
}
License
Apache-2.0
- License
- apache-2.0
- Languages
- en
- Created
- Mar 15, 2024
- Updated
- Mar 19, 2024