Downloads · 30 days
31
1% of all-time downloads
mesolitica/translation-t5-tiny-standard-bahasa-cased
translation-t5-tiny-standard-bahasa-cased is a machine learning model from mesolitica. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
Trained on 1536 context length, able to translate malay, pasar malay (social media texts or local context), english, manglish, javanese, banjarese and indonesian to target language. It also able to maintain the text s…
Downloads · 30 days
31
1% of all-time downloads
All-time downloads
3.5K
Public
Parameters
34.8M
904 MB on disk
Likes
1
Public
Click a slice to open those files.
.bin139 MB · 66%
From the Hugging Face model README
Trained on 1536 context length, able to translate malay, pasar malay (social media texts or local context), english, manglish, javanese, banjarese and indonesian to target language. It also able to maintain the text structure as it is and only translate necessary texts, eg, programming code.
Try it at https://huggingface.co/spaces/mesolitica/malaysian-translation
from transformers import T5ForConditionalGeneration, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
'mesolitica/translation-t5-tiny-standard-bahasa-cased',
use_fast=False
)
model = T5ForConditionalGeneration.from_pretrained(
'mesolitica/translation-t5-tiny-standard-bahasa-cased'
)
s = 'Hai, ada yang bisa saya bantu?'
input_ids = tokenizer.encode(f'terjemah ke Melayu: {s}', return_tensors = 'pt')
outputs = model.generate(input_ids, max_length = 100)
all_special_ids = [0, 1, 2]
outputs = [i for i in outputs[0] if i not in all_special_ids]
print(tokenizer.decode(outputs, spaces_between_special_tokens = False))