Downloads · 30 days
0
dravida_tokenizers_combined
jaswanthrk/dravida_tokenizers_combined
dravida_tokenizers_combined is a machine learning model from jaswanthrk. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
posteriorKaTeMaTallamallama.model - This is SP format tokenizer obtained by merging Kannada, Telugu, Malayalam, Tamil and Llama-2 tokenizers.
Access
Public
Updated Mar 24, 2024
Repo size
1.8 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.model1.8 MB · 80%
At a glance
- License
- mit
- Access
- Public
- Created
- Mar 24, 2024
- Updated
- Mar 24, 2024
- SHA
- 0ddb6ec5
Model card
From the Hugging Face model README
posterior_KaTeMaTa_llama_llama.model
- This is SP format tokenizer obtained by merging Kannada, Telugu, Malayalam, Tamil and Llama-2 tokenizers.
posterior_dr_llama_15_32k_balanced.model posterior_dr_llama_15_32k_balanced.vocab
- These is SP format tokenizer obtained by training the SP tokenizer using the four languages data.
- License
- mit
- Created
- Mar 24, 2024
- Updated
- Mar 24, 2024