Downloads · 30 days
169
0% of all-time downloads
FusionNet_7Bx2_MoE_14B
TomGrc/FusionNet_7Bx2_MoE_14B
FusionNet_7Bx2_MoE_14B is a text generation model from TomGrc. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Fine-tuned model on English language using MoE method. The FusionNet is a model to experiment with the MoE method, which could significantly increase the performance of the original model. The FusionNet has 12.9B para…
All-time downloads
40.2K
Public
Parameters
12.9B
25.8 GB on disk
Likes
36
Public
Repo makeup
Click a slice to open those files.
.safetensors25.8 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- mixtral
- Access
- Public
- Created
- Jan 12, 2024
- Updated
- Mar 4, 2024
- SHA
- e4076fa3
Try a prompt
Model card
From the Hugging Face model README
FusionNet
Fine-tuned model on English language using MoE method.
Model description
The FusionNet is a model to experiment with the MoE method, which could significantly increase the performance of the original model. The FusionNet has 12.9B parameters, and this model is fine-tuned. Enjoy!
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 75.91 |
| AI2 Reasoning Challenge (25-Shot) | 73.55 |
| HellaSwag (10-Shot) | 88.84 |
| MMLU (5-Shot) | 64.68 |
| TruthfulQA (0-shot) | 69.60 |
| Winogrande (5-shot) | 88.16 |
| GSM8k (5-shot) | 70.66 |
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- mit
- Languages
- en
- Created
- Jan 12, 2024
- Updated
- Mar 4, 2024