Downloads · 30 days
0
gmteacher/simple-word-tokenizer
simple-word-tokenizer is a machine learning model from gmteacher. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
Downloads · 30 days
0
Access
Public
Updated Oct 16, 2024
Repo size
—
Likes
0
Public
Click a slice to open those files.
.json1.6 KB · 50%
From the Hugging Face model README
교육 목적의 simple word tokenizer입니다.