Downloads · 30 days
13
0% of all-time downloads
Goku-8x22B-v0.2
MaziyarPanahi/Goku-8x22B-v0.2
Goku-8x22B-v0.2 is a text generation model from MaziyarPanahi. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
<img src="./Goku-8x22b-v0.1.webp" alt="Goku 8x22B v0.1 Logo" width="500" style="margin-left:'auto' margin-right:'auto' display:'block'"/
All-time downloads
6K
Public
Parameters
141B
562 GB on disk
Likes
6
Public
Repo makeup
Click a slice to open those files.
.safetensors562 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- mixtral
- Access
- Public
- Created
- Apr 16, 2024
- Updated
- Apr 16, 2024
- SHA
- 215a4d7c
Base models
Model card
From the Hugging Face model README
<img src="./Goku-8x22b-v0.1.webp" alt="Goku 8x22B v0.1 Logo" width="500" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
Goku-8x22B-v0.2 (Goku 141b-A35b)
A fine-tuned version of v2ray/Mixtral-8x22B-v0.1 model on the following datasets:
- teknium/OpenHermes-2.5
- WizardLM/WizardLM_evol_instruct_V2_196k
- microsoft/orca-math-word-problems-200k
This model has a total of 141b parameters with 35b only active. The major difference in this version is that the model was trained on more datasets and with an 8192 sequence length. This results in the model being able to generate longer and more coherent responses.
How to use it
Use a pipeline as a high-level helper:
from transformers import pipeline
pipe = pipeline("text-generation", model="MaziyarPanahi/Goku-8x22B-v0.2")
Load model directly:
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("MaziyarPanahi/Goku-8x22B-v0.2")
model = AutoModelForCausalLM.from_pretrained("MaziyarPanahi/Goku-8x22B-v0.2")
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- apache-2.0
- Languages
- fr, it, de, es, en
- Created
- Apr 16, 2024
- Updated
- Apr 16, 2024