Downloads · 30 days
57
41% of all-time downloads
amharic-sentencepiece-tokenizer
gashudemman/amharic-sentencepiece-tokenizer
amharic-sentencepiece-tokenizer is a machine learning model from gashudemman. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This repository contains a SentencePiece tokenizer trained on a large corpus of Amharic text.
All-time downloads
139
Public
Repo size
278 KB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.model278 KB · 99%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Aug 6, 2024
- Updated
- Aug 7, 2024
- SHA
- 4318811f
Model card
From the Hugging Face model README
Amharic SentencePiece Tokenizer
This repository contains a SentencePiece tokenizer trained on a large corpus of Amharic text.
Usage
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("your_username/amharic-sentencepiece-tokenizer")
text = "ይህ አማርኛ ጽሑፍ ነው።"
encoded_text = tokenizer(text)
print(encoded_text)
- License
- apache-2.0
- Created
- Aug 6, 2024
- Updated
- Aug 7, 2024