Downloads · 30 days
15
1% of all-time downloads
fairseq-moe-15b-bf16
pingzhili/fairseq-moe-15b-bf16
fairseq-moe-15b-bf16 is a text generation model from pingzhili. Use it when you need the model to write or continue text. It is set up for transformers.
This is a Hugging Face transformers-style conversion of the original SMoE 15B-parameter model with BFLOAT16 from the paper "Efficient Large Scale Language Modeling with Mixtures of Experts" from Artetxe et al. The ori…
All-time downloads
1.4K
Public
Repo size
58.4 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.bin29.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Access
- Public
- Created
- Sep 24, 2023
- Updated
- Sep 24, 2023
- SHA
- 582ee1e2
Try a prompt
Model card
From the Hugging Face model README
This is a Hugging Face transformers-style conversion of the original SMoE 15B-parameter model with BFLOAT16 from the paper "Efficient Large Scale Language Modeling with Mixtures of Experts" from Artetxe et al. The original model card can be found at https://github.com/facebookresearch/fairseq/blob/main/examples/moe_lm/model_card.md.
The usage example and modeling code can be found at https://github.com/pingzhili/light-fairseq
- Task
- Text Generation
- Library
- transformers
- Languages
- en
- Created
- Sep 24, 2023
- Updated
- Sep 24, 2023