Downloads · 30 days
0
BPE_GPT2_TinyStoriesV2_cleaned_1024
fhswf/BPE_GPT2_TinyStoriesV2_cleaned_1024
BPE_GPT2_TinyStoriesV2_cleaned_1024 is a machine learning model from fhswf. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
BPE Tokenizer Model for dataset 'fhswf/TinyStoriesV2cleaned'
Access
Public
Updated May 23, 2024
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json43.9 KB · 86%
At a glance
- License
- mit
- Access
- Public
- Created
- May 17, 2024
- Updated
- May 23, 2024
- SHA
- b45800de
Datasets
Model card
From the Hugging Face model README
BPE_GPT2_TinyStoriesV2_cleaned
BPE Tokenizer Model for dataset 'fhswf/TinyStoriesV2_cleaned'
Based on get-neo BPE Tokenizer, but with a smaller vocabulary. Trained with TinyStoriesV2.
- Vocab Size: 1024
- 256 Base chars
- 1 extra Token: <|endoftext|>
- 767 merges
- License
- mit
- Languages
- en
- Created
- May 17, 2024
- Updated
- May 23, 2024