transformersbook

codeparrot-small-vocabulary

transformersbook/codeparrot-small-vocabulary

codeparrot-small-vocabulary is a machine learning model from transformersbook. Use it for the machine learning task on the model card, and read the license before you ship it in a product.

This is a small version of the CodeParrot tokenizer trained on the CodeParrot Python code dataset. The tokenizer is trained in Chapter 10: Training Transformers from Scratch in the NLP with Transformers book. You can…