Downloads · 30 days
40
3% of all-time downloads
xlm-align
CZWin32768/xlm-align
xlm-align is a fill-mask model from CZWin32768. Use it when you need the model to fill a missing word. It is set up for transformers.
Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment (ACL-2021, paper, github)
All-time downloads
1.2K
Public
Repo size
1.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin942 MB · 99%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- Model type
- xlm-roberta
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Jul 21, 2021
- SHA
- 6b7a0812
Try a prompt
Model card
From the Hugging Face model README
XLM-Align
Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment (ACL-2021, paper, github)
XLM-Align is a pretrained cross-lingual language model that supports 94 languages. See details in our paper.
Example
model = = AutoModel.from_pretrained("CZWin32768/xlm-align")
Evaluation Results
XTREME cross-lingual understanding tasks:
| Model | POS | NER | XQuAD | MLQA | TyDiQA | XNLI | PAWS-X | Avg |
|---|---|---|---|---|---|---|---|---|
| XLM-R_base | 75.6 | 61.8 | 71.9 / 56.4 | 65.1 / 47.2 | 55.4 / 38.3 | 75.0 | 84.9 | 66.4 |
| XLM-Align | 76.0 | 63.7 | 74.7 / 59.0 | 68.1 / 49.8 | 62.1 / 44.8 | 76.2 | 86.8 | 68.9 |
MD5
b9d214025837250ede2f69c9385f812c config.json
6005db708eb4bab5b85fa3976b9db85b pytorch_model.bin
bf25eb5120ad92ef5c7d8596b5dc4046 sentencepiece.bpe.model
eedbd60a7268b9fc45981b849664f747 tokenizer.json
About
Contact: [email protected]
BibTeX:
@article{xlmalign,
title={Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment},
author={Zewen Chi and Li Dong and Bo Zheng and Shaohan Huang and Xian-Ling Mao and Heyan Huang and Furu Wei},
journal={arXiv preprint arXiv:2106.06381},
year={2021}
}
- Task
- Fill-Mask
- Library
- transformers
- Type
- xlm-roberta
- Created
- Mar 2, 2022
- Updated
- Jul 21, 2021