Downloads · 30 days
8
0% of all-time downloads
chinesebabylm-control-adamw-bert-mlm-9p53m
LikC1606/chinesebabylm-control-adamw-bert-mlm-9p53m
chinesebabylm-control-adamw-bert-mlm-9p53m is a fill-mask model from LikC1606. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as other.
Small from-scratch BERT-style masked language model trained as the Round 20 DeepScientist control-arm AdamW no-bias-correction pilot.
All-time downloads
3K
Public
Parameters
9.5M
38.1 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors38.1 MB · 100%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- other
- Model type
- bert
- Access
- Public
- Created
- May 10, 2026
- Updated
- May 10, 2026
- SHA
- 012c88a5
Try a prompt
Model card
From the Hugging Face model README
ChineseBabyLM Control AdamW BERT MLM 9.53M
Small from-scratch BERT-style masked language model trained as the Round 20 DeepScientist control-arm AdamW no-bias-correction pilot.
Model Details
- Architecture:
BertForMaskedLM - Model type: masked language model / encoder-only BERT
- Parameters: 9,527,714
- Layers: 4
- Attention heads: 6
- Hidden size: 384
- Vocabulary size: 5,666
- Max position embeddings: 256
Training Data
Chinese BabyLM official 10k stratified sample; control-arm Round 20 AdamW no-bias-correction pilot.
Evaluation
Official zero-shot evaluator results:
| Task | Accuracy |
|---|---|
| ZhoBLiMP | 59.74 |
| Hanzi structure | 47.00 |
| Hanzi pinyin | 31.00 |
| Mean available tasks | 45.91 |
Package Source
This model was packaged from control_round20_adamw_no_bias_correction_bert_mlm_9p53m in the local ChineseBabyLM best-model archive.
- Task
- Fill-Mask
- Library
- transformers
- Type
- bert
- License
- other
- Languages
- zh
- Created
- May 10, 2026
- Updated
- May 10, 2026