Downloads · 30 days
17
3% of all-time downloads
zzzotop/zero-shot-cross-lingual-transfer-demo-masked
zero-shot-cross-lingual-transfer-demo-masked is a fill-mask model from zzzotop. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as apache-2.0.
Next word prediction in 103 languages. Give it a sentence in another language, and replace one of the words with "[MASK]". Works with English too, obviously, but that defeats the point of the demo.
Downloads · 30 days
17
3% of all-time downloads
All-time downloads
630
Public
Repo size
21.7 GB
Likes
0
Public
Click a slice to open those files.
.bin542 MB · 99%
From the Hugging Face model README
Next word prediction in 103 languages. Give it a sentence in another language, and replace one of the words with "[MASK]". Works with English too, obviously, but that defeats the point of the demo.
distilbert-base-multilingual-cased finetuned on 50,000 examples from r/explainlikeimfive subset of ELI5 dataset, for English causal language modelling. All knowledge of target languages is acquired from pretraining only.
Hyperparameters: epochs 3, learning rate 2e-5, batch size 8, weight decay 0.01, optimizer Adam with betas=(0.9,0.999) and epsilon=1e-08.
Final model perplexity 10.22