Downloads · 30 days
17
6% of all-time downloads
DMoE-Bloom-560M-16Experts-128Langs
chongli17/DMoE-Bloom-560M-16Experts-128Langs
DMoE-Bloom-560M-16Experts-128Langs is a machine learning model from chongli17. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as bigscience-bloom-rail-1.0.
The model is extended from Bloom-560m and continues to train 17.7B tokens on the 128 languages text of MADLAD-400.
All-time downloads
269
Public
Parameters
1.7B
3.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.4 GB · 100%
At a glance
- License
- bigscience-bloom-rail-1.0
- Model type
- bloommoedyn
- Access
- Public
- Created
- May 28, 2025
- Updated
- May 28, 2025
- SHA
- c70eaeee
Model card
From the Hugging Face model README
Model Card for DMoE-Bloom-560M-16Experts-128Langs
The model is extended from Bloom-560m and continues to train 17.7B tokens on the 128 languages text of MADLAD-400.
Code
The code used to train this model refers to the github repo.
Citation
@inproceedings{li-etal-2025-DMoE,
author = {Chong Li and
Yingzhuo Deng and
Jiajun Zhang and
Chengqing Zong},
title = "Group then Scale: Dynamic Mixture-of-Experts Multilingual Language Model",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
}
- Type
- bloommoedyn
- License
- bigscience-bloom-rail-1.0
- Created
- May 28, 2025
- Updated
- May 28, 2025