Downloads · 30 days
17
4% of all-time downloads
sf_ip_babylm_1
omarmomen/sf_ip_babylm_1
sf_ip_babylm_1 is a fill-mask model from omarmomen. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as mit.
This model is part of the experiments in my master's thesis titled "Linguistic Structure Induction from Language Models" (https://arxiv.org/abs/2403.09714).
All-time downloads
416
Public
Repo size
332 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin166 MB · 99%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- mit
- Model type
- structformer_in_parser
- Access
- Public
- Created
- Feb 4, 2024
- Updated
- Mar 26, 2024
- SHA
- 31181e7d
Try a prompt
Datasets
Model card
From the Hugging Face model README
Model Card for omarmomen/sf_ip_babylm_1
This model is part of the experiments in my master's thesis titled "Linguistic Structure Induction from Language Models" (https://arxiv.org/abs/2403.09714).
"omarmomen/sf_ip_babylm_1" is the StructFormer (SF_m=4) referred to in Chapter 5 (p. 59); it is an in-between parser variant with the parser network positioned after 4 transformer blocks.
The model is trained on the BabyLM 10M dataset, with a RobertaTokenizer pretrained on the BabyLM 10M dataset with 16K tokens (https://huggingface.co/omarmomen/babylm_bpe_tokenizer_16k).
- Task
- Fill-Mask
- Library
- transformers
- Type
- structformer_in_parser
- License
- mit
- Languages
- en
- Created
- Feb 4, 2024
- Updated
- Mar 26, 2024