Downloads · 30 days
20
6% of all-time downloads
Mistral-29b-Merge-Base
Vezora/Mistral-29b-Merge-Base
Mistral-29b-Merge-Base is a text generation model from Vezora. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
All-time downloads
337
Public
Parameters
28.2B
56.4 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors56.4 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- mistral
- Access
- Public
- Created
- Nov 3, 2023
- Updated
- Nov 4, 2023
- SHA
- 6a86a7c1
Try a prompt
Model card
From the Hugging Face model README
Contributors: Nicolas Mejia Petit
Mistral 29b: A New Base Model
The objective of this model is to serve as a new fully open source base model with 29.2 billion parameters.
This model spits out jargon, and needs to be fine tuned, either with qlora, with the adapter attached to every layer, or better yet a full fine tune.
Model Creation
The model was created by stacking four models: Dolphin, Zephyr, Meta-math7b, and Speechless code, to form a single model.
Useful Resources
- Model Merging (MergeKit) (https://github.com/cg123/mergekit)
Source Models
- Dolphin2.1-mistral-7b by Eric Hartford (https://huggingface.co/ehartford/dolphin-2.1-mistral-7b)
- Zephyr-7b-beta by HuggingFace (https://huggingface.co/HuggingFaceH4/zephyr-7b-beta)
- MetaMath-Mistral-7B by meta-math (https://huggingface.co/meta-math/MetaMath-Mistral-7B)
- speechless-code-mistral-7b-v1.0 (https://huggingface.co/uukuguy/speechless-code-mistral-7b-v1.0)
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- License
- apache-2.0
- Created
- Nov 3, 2023
- Updated
- Nov 4, 2023