Downloads · 30 days
15
4% of all-time downloads
sf_babylm_1
omarmomen/sf_babylm_1
sf_babylm_1 is a fill-mask model from omarmomen. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as mit.
This model is part of the experiments in my master's thesis titled "Linguistic Structure Induction from Language Models" (https://arxiv.org/abs/2403.09714).
All-time downloads
347
Public
Repo size
10.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin7.2 GB · 52%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- mit
- Model type
- structformer
- Access
- Public
- Created
- Feb 3, 2024
- Updated
- Mar 26, 2024
- SHA
- a64dd667
Try a prompt
Datasets
Model card
From the Hugging Face model README
Model Card for omarmomen/sf_babylm_1
This model is part of the experiments in my master's thesis titled "Linguistic Structure Induction from Language Models" (https://arxiv.org/abs/2403.09714).
"omarmomen/sf_babylm_1" is the StructFormer (SF_m=0) referred to in Chapter 5 (p. 59), where the parser network is position ahead of all the attention blocks.
The model is trained on the BabyLM 10M dataset, with a RobertaTokenizer pretrained on the BabyLM 10M dataset with 16K tokens (https://huggingface.co/omarmomen/babylm_bpe_tokenizer_16k).
- Task
- Fill-Mask
- Library
- transformers
- Type
- structformer
- License
- mit
- Languages
- en
- Created
- Feb 3, 2024
- Updated
- Mar 26, 2024