Downloads · 30 days
0
qwen3-14b-myanmar-tokenizer
kkomyoeminaung/qwen3-14b-myanmar-tokenizer
qwen3-14b-myanmar-tokenizer is a machine learning model from kkomyoeminaung. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
Access
Public
Updated Jul 6, 2026
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json5.3 MB · 100%
At a glance
- License
- other
- Access
- Public
- Created
- Jul 5, 2026
- Updated
- Jul 6, 2026
- SHA
- 94ada488
Model card
From the Hugging Face model README
Qwen Myanmar Tokenizer v4
Metrics
| Metric | Value |
|---|---|
| Vocab Size | 65,536 |
| Myanmar Tokens | 56,341 (85.97%) |
| Compression | 0.1718 |
| Avg Token Length | 5.83 |
| Coverage | 54.18% |
| Bits/Byte | 2.5412 |
Comparison with Original Qwen
| Metric | Original Qwen | Myanmar | Improvement |
|---|---|---|---|
| Compression | 1.4607 | 0.1718 | +88.2% |
| Coverage | 3.11% | 54.18% | +1639.5% |
Usage
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("kkomyoeminaung/qwen3-14b-myanmar-tokenizer")
tokens = tokenizer.tokenize("မြန်မာနိုင်ငံရဲ့ မြို့တော်က နေပြည်တော်ပါ။")
print(tokens)
- License
- other
- Languages
- my
- Created
- Jul 5, 2026
- Updated
- Jul 6, 2026