minpeter

tiny-ko-tokenizer-32k-250718

minpeter/tiny-ko-tokenizer-32k-250718

tiny-ko-tokenizer-32k-250718 is a machine learning model from minpeter. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.

trained on a minpeter/tiny-ko-corpus full, HuggingFaceTB/smollm-corpus subset of fineweb-edu-dedup, comopedia-v2