Downloads · 30 days
24
8% of all-time downloads
Mistral_7B_Dolphin2.1_LIMA0.5_fp16
RossAscends/Mistral_7B_Dolphin2.1_LIMA0.5_fp16
Mistral_7B_Dolphin2.1_LIMA0.5_fp16 is a text generation model from RossAscends. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
ehartford's merge of Mistral 7B 0.1 with his Dolphin 2.1 dataset
All-time downloads
314
Public
Parameters
7.2B
14.5 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors14.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- mistral
- Access
- Public
- Created
- Oct 27, 2023
- Updated
- Oct 28, 2023
- SHA
- 28a26dd6
Try a prompt
Model card
From the Hugging Face model README
ehartford's merge of Mistral 7B 0.1 with his Dolphin 2.1 dataset
https://huggingface.co/ehartford/dolphin-2.1-mistral-7b
and
LIMA RP dataset applied as a lora at 0.5 weight
https://huggingface.co/lemonilia/limarp-llama2-v2/
Purpose of the model is to be RP-focused, smart, fast, and lightweight for users with low VRAM.
I've already built the exl2 4bpw quant (linked below), and it will run 8k ctx at around 6GB VRAM and respond to a full context at roughly 30tps (tested on my 3060) if exl2_hf loader is used with FA2 enabled.
Model has been tested by several users on the SillyTavern discord server, and run on Horde for a full day - with good results.
https://huggingface.co/RossAscends/Mistral7B_Dolphin2.1_LIMARP0.5_4bpw_exl2
Mistral or ChatML context presets both possible.
exllama v2 4bpw quant: https://huggingface.co/RossAscends/Mistral7B_Dolphin2.1_LIMARP0.5_4bpw_exl2
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- License
- mit
- Created
- Oct 27, 2023
- Updated
- Oct 28, 2023