Downloads · 30 days
0
GPT2-PretrainV1-Tokenizer-en
terrycraddock/GPT2-PretrainV1-Tokenizer-en
GPT2-PretrainV1-Tokenizer-en is a machine learning model from terrycraddock. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Custom tokenizer used to tokenize my dataset terrycraddock/GPT2-PretrainV1-en. This tokenizer is basically the default gpt2 tokenizer except I added a [PAD] token for the trainer to perform distilled knowledge trainin…
Access
Public
Updated Dec 31, 2023
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json1.1 MB · 86%
At a glance
- License
- mit
- Access
- Public
- Created
- Dec 31, 2023
- Updated
- Dec 31, 2023
- SHA
- c8bc9ccf
Model card
From the Hugging Face model README
Custom tokenizer used to tokenize my dataset terrycraddock/GPT2-PretrainV1-en. This tokenizer is basically the default gpt2 tokenizer except I added a [PAD] token for the trainer to perform distilled knowledge training as well as trained this on my dataset to be more efficent.
Link to tokenized dataset: https://huggingface.co/datasets/terrycraddock/GPT2-PretrainV1-Tokenized-en
Link to non-tokenized dataset: https://huggingface.co/datasets/terrycraddock/GPT2-PretrainV1-en
This customer tokenizer as well as the datasets mentioned are intended to be used as a pretrain corpus for knowledge distillation learning from a larger GPT2 model to a smaller custom one.
- License
- mit
- Created
- Dec 31, 2023
- Updated
- Dec 31, 2023