Downloads · 30 days
0
spanish-tweet-tokenizer
jcollado/spanish-tweet-tokenizer
spanish-tweet-tokenizer is a machine learning model from jcollado. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This tokenizer has been trained with tweets that have been preprocessed as follows:
Access
Public
Updated Mar 23, 2022
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json651 KB · 75%
At a glance
- Access
- Public
- Created
- Mar 23, 2022
- Updated
- Mar 23, 2022
- SHA
- 1756e415
Model card
From the Hugging Face model README
Text preprocessing
This tokenizer has been trained with tweets that have been preprocessed as follows:
- User mentions (@user_name) have been replaced with the word user.
- URLs have been replace with the word url.
- WIP. If you are going to use this tokenizer, we recommend you to preprocess your own dataset in the same manner.
- Created
- Mar 23, 2022
- Updated
- Mar 23, 2022