Downloads · 30 days
26
1% of all-time downloads
flax-bigbird-natural-questions
vasudevgupta/flax-bigbird-natural-questions
flax-bigbird-natural-questions is a machine learning model from vasudevgupta. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
This checkpoint is obtained after training FlaxBigBirdForQuestionAnswering (with extra pooler head) on naturalquestions dataset on TPU v3-8. This dataset takes around ~100 GB on disk. But thanks to Cloud TPUs and Jax,…
All-time downloads
3K
Public
Repo size
8.5 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.msgpack529 MB · 88%
At a glance
- Library
- transformers
- License
- apache-2.0
- Model type
- big_bird
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Dec 31, 2022
- SHA
- 9349ed10
Try a prompt
Datasets
Spaces
Model card
From the Hugging Face model README
This checkpoint is obtained after training FlaxBigBirdForQuestionAnswering (with extra pooler head) on natural_questions dataset on TPU v3-8. This dataset takes around ~100 GB on disk. But thanks to Cloud TPUs and Jax, each epoch took just 4.5 hours. Script for training can be found here: https://github.com/vasudevgupta7/bigbird
Use this model just like any other model from 🤗Transformers
from transformers import FlaxBigBirdForQuestionAnswering, BigBirdTokenizerFast
model_id = "vasudevgupta/flax-bigbird-natural-questions"
model = FlaxBigBirdForQuestionAnswering.from_pretrained(model_id)
tokenizer = BigBirdTokenizerFast.from_pretrained(model_id)
In case you are interested in predicting category (null, long, short, yes, no) as well, use FlaxBigBirdForNaturalQuestions (instead of FlaxBigBirdForQuestionAnswering) from my training script.
| Exact Match | 55.12 |
|---|
Evaluation script: https://colab.research.google.com/github/vasudevgupta7/bigbird/blob/main/notebooks/evaluate-flax-natural-questions.ipynb
- Library
- transformers
- Type
- big_bird
- License
- apache-2.0
- Languages
- en
- Created
- Mar 2, 2022
- Updated
- Dec 31, 2022