Downloads · 30 days
8.7K
4% of all-time downloads
LogoS-7Bx2-MoE-13B-v0.2
RubielLabarta/LogoS-7Bx2-MoE-13B-v0.2
LogoS-7Bx2-MoE-13B-v0.2 is a text generation model from RubielLabarta. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Model built by @RubielLabarta using SLERP merge method. The model is release for research purposes only, commercial use is not allowed.
All-time downloads
200K
Public
Parameters
12.9B
25.8 GB on disk
Likes
10
Public
Repo makeup
Click a slice to open those files.
.safetensors25.8 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- mixtral
- Access
- Public
- Created
- Jan 21, 2024
- Updated
- Mar 14, 2024
- SHA
- fb0f72b9
Try a prompt
Base models
Model card
From the Hugging Face model README
LogoS-7Bx2-MoE-13B-v0.1
Model built by @RubielLabarta using SLERP merge method. The model is release for research purposes only, commercial use is not allowed.
The LogoS is a model to experiment with the MoE method, which could significantly increase the performance of the original model. The model has 12.9B parameters.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 77.14 |
| AI2 Reasoning Challenge (25-Shot) | 74.49 |
| HellaSwag (10-Shot) | 89.07 |
| MMLU (5-Shot) | 64.74 |
| TruthfulQA (0-shot) | 74.57 |
| Winogrande (5-shot) | 88.32 |
| GSM8k (5-shot) | 71.65 |
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- apache-2.0
- Languages
- en, es
- Created
- Jan 21, 2024
- Updated
- Mar 14, 2024