Downloads · 30 days
14
4% of all-time downloads
str33t/dhivehi-byt5-latin2thaana-keymap-v1
dhivehi-byt5-latin2thaana-keymap-v1 is a translation model from str33t. Use it when you need text moved from one language to another. It is set up for transformers. The card lists the license as apache-2.0.
ByT5 fine-tune for Latin → Dhivehi transliteration that emits an ASCII phonetic keymap instead of Thaana UTF-8 bytes. A short post-processing function maps the ASCII output back to Thaana script.
Downloads · 30 days
14
4% of all-time downloads
All-time downloads
337
Public
Parameters
300M
2.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.2 GB · 100%
From the Hugging Face model README
ByT5 fine-tune for Latin → Dhivehi transliteration that emits an ASCII phonetic keymap instead of Thaana UTF-8 bytes. A short post-processing function maps the ASCII output back to Thaana script.
Because Thaana codepoints (U+0780–U+07BF) occupy three UTF-8 bytes and ByT5 generates one decoder step per output byte, decoding 100 Thaana characters costs 300 decoder steps. Emitting the keymap form (1 byte per character) and converting after generate() finishes gives an end-user-equivalent result at ~3× lower decoder cost.
The model warm-starts from Neobe/dhivehi-byt5-latin2thaana-v1; only the decoder retargets to the new output alphabet.
The model's raw output is ASCII (Segha phonetic keyboard layout). You need the keymap_to_thaana conversion function from the project repo to render Thaana.
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from keymap import keymap_to_thaana # from div-transliteration repo
MODEL_ID = "str33t/dhivehi-byt5-latin2thaana-keymap-v1"
tok = AutoTokenizer.from_pretrained(MODEL_ID)
mdl = AutoModelForSeq2SeqLM.from_pretrained(MODEL_ID)
text = "aharumen anekkaa ves bahdhalu vee"
inputs = tok(text, return_tensors="pt")
out = mdl.generate(**inputs, num_beams=4, max_new_tokens=512)
ascii_keymap = tok.decode(out[0], skip_special_tokens=True)
thaana = keymap_to_thaana(ascii_keymap)
print(thaana)
The Segha phonetic layout maps every Thaana letter / fili / sukun to a unique ASCII byte (see keymap.py in the project repo for the exact tables, taken verbatim from jawish/jtk). Conversion is deterministic and lossless on in-layout characters; digits, punctuation, and whitespace pass through.
Neobe/dhivehi-byt5-latin2thaana-v1alakxender/dhivehi-transliteration-pairs (Thaana side converted to keymap via keymap.py)MODEL_NOTES.md.Full methodology, ablations, and results: project README.
Apache 2.0 (inherited from the base model and ByT5).