Downloads · 30 days
609
0% of all-time downloads
fairseq-dense-125M
KoboldAI/fairseq-dense-125M
fairseq-dense-125M is a text generation model from KoboldAI. Use it when you need the model to write or continue text. It is set up for transformers.
This is a Hugging Face transformers-compatible conversion of the original dense 125M-parameter model from the paper "Efficient Large Scale Language Modeling with Mixtures of Experts" from Artetxe et al. Please refer t…
All-time downloads
378K
Public
Parameters
162M
649 MB on disk
Likes
5
Public
Repo makeup
Click a slice to open those files.
.bin325 MB · 50%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- xglm
- Access
- Public
- Created
- Mar 2, 2022
- Updated
- Nov 18, 2023
- SHA
- 7ed81ed9
Try a prompt
Model card
From the Hugging Face model README
This is a Hugging Face transformers-compatible conversion of the original dense 125M-parameter model from the paper "Efficient Large Scale Language Modeling with Mixtures of Experts" from Artetxe et al. Please refer to the original model card, which can be found at https://github.com/facebookresearch/fairseq/blob/main/examples/moe_lm/model_card.md.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 26.0 |
| ARC (25-shot) | 24.06 |
| HellaSwag (10-shot) | 34.14 |
| MMLU (5-shot) | 23.98 |
| TruthfulQA (0-shot) | 43.72 |
| Winogrande (5-shot) | 50.59 |
| GSM8K (5-shot) | 0.0 |
| DROP (3-shot) | 5.5 |
- Task
- Text Generation
- Library
- transformers
- Type
- xglm
- Languages
- en
- Created
- Mar 2, 2022
- Updated
- Nov 18, 2023