Downloads · 30 days
0
babylm-bpe-50k
augustinian-babylm/babylm-bpe-50k
babylm-bpe-50k is a machine learning model from augustinian-babylm. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
Byte-level BPE tokenizer (GPT-2/RoBERTa-style) for DeBERTa pretraining on the BabyLM corpus. Realised vocab size: 50,000. Built for the latest transformers.
Access
Public
Updated May 25, 2026
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json3.6 MB · 100%
At a glance
- Library
- transformers
- Access
- Public
- Created
- May 25, 2026
- Updated
- May 25, 2026
- SHA
- fe33c5b5
Model card
From the Hugging Face model README
augustinian-babylm/babylm-bpe-50k
Byte-level BPE tokenizer (GPT-2/RoBERTa-style) for DeBERTa pretraining on the
BabyLM corpus. Realised vocab size: 50,000. Built for the latest
transformers.
Special tokens: [PAD]=0 [CLS]=1 [SEP]=2 [MASK]=3 [UNK]=4.
Sequences are auto-wrapped as [CLS] ... [SEP].
Trained on: https://raw.githubusercontent.com/Leukas/babylm25/main/data/bb24.train.
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("augustinian-babylm/babylm-bpe-50k")
# DeBERTa config: config.vocab_size = tok.vocab_size
- Library
- transformers
- Created
- May 25, 2026
- Updated
- May 25, 2026