Downloads · 30 days
80
1% of all-time downloads
FusionNet_7Bx2_MoE_v0.1
TomGrc/FusionNet_7Bx2_MoE_v0.1
FusionNet_7Bx2_MoE_v0.1 is a text generation model from TomGrc. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Fine-tuned model on English language using MoE method. The improved version from FusionNet7Bx2MoE14B. The FusionNet7Bx2MoEv0.1 is a model to experiment with the MoE method, which could significantly increase the perfo…
All-time downloads
13K
Public
Parameters
12.9B
25.8 GB on disk
Likes
6
Public
Repo makeup
Click a slice to open those files.
.safetensors25.8 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- mixtral
- Access
- Public
- Created
- Jan 30, 2024
- Updated
- Mar 4, 2024
- SHA
- 9a25c33c
Try a prompt
Model card
From the Hugging Face model README
FusionNet_7Bx2_MoE_v0.1
Fine-tuned model on English language using MoE method. The improved version from FusionNet_7Bx2_MoE_14B.
Model description
The FusionNet_7Bx2_MoE_v0.1 is a model to experiment with the MoE method, which could significantly increase the performance of the original model. The FusionNet has 12.9B parameters, and this model is fine-tuned. Enjoy!
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 76.16 |
| AI2 Reasoning Challenge (25-Shot) | 74.06 |
| HellaSwag (10-Shot) | 88.90 |
| MMLU (5-Shot) | 65.00 |
| TruthfulQA (0-shot) | 71.20 |
| Winogrande (5-shot) | 87.53 |
| GSM8k (5-shot) | 70.28 |
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- mit
- Languages
- en
- Created
- Jan 30, 2024
- Updated
- Mar 4, 2024