bert-uncased-L2-H256-A4

lyeonii/bert-uncased-L2-H256-A4

bert-uncased-L2-H256-A4 is a machine learning model from lyeonii. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.

This is one of 24 smaller BERT models (English only, uncased, trained with WordPiece masking) released by google-research/bert.

Downloads · 30 days

2% of all-time downloads

All-time downloads

402

Public

Repo size

77.5 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin38.8 MB · 99%

At a glance

Library: transformers
License: mit
Model type: bert
Access: Public
Created: Apr 20, 2023
Updated: Apr 20, 2023
SHA: 390354a4

Model card

From the Hugging Face model README

bert-uncased-L2-H256-A4

This is one of 24 smaller BERT models (English only, uncased, trained with WordPiece masking) released by google-research/bert.

These BERT models was released as TensorFlow checkpoints, however, this is the converted version to PyTorch. More information can be found in google-research/bert or lyeoni/convert-tf-to-pytorch.

Evaluation

Here are the evaluation scores (F1/Accuracy) for the MPRC task.

Model	MRPC
BERT-Tiny	81.22/68.38
BERT-Mini	81.43/69.36
BERT-Small	81.41/70.34
BERT-Medium	83.33/73.53
BERT-Base	85.62/78.19

References

@article{turc2019,
  title={Well-Read Students Learn Better: On the Importance of Pre-training Compact Models},
  author={Turc, Iulia and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina},
  journal={arXiv preprint arXiv:1908.08962v2 },
  year={2019}
}

Library: transformers
Type: bert
License: mit
Languages: en
Created: Apr 20, 2023
Updated: Apr 20, 2023