aeonium

tokenizer-64k

aeonium/tokenizer-64k

tokenizer-64k is a machine learning model from aeonium. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.