Downloads · 30 days
12
1% of all-time downloads
Mistral-7B-Instruct-v0.1-8bit
LsTam/Mistral-7B-Instruct-v0.1-8bit
Mistral-7B-Instruct-v0.1-8bit is a text generation model from LsTam. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
The Mistral-7B-Instruct-v0.1-8bit is a 8bit quantize version with torchdtype=torch.float16, I just load in 8bit and push here Mistral-7B-Instruct-v0.1.
All-time downloads
900
Public
Parameters
7.2B
7.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors7.5 GB · 100%
Parameter types
How the weights are stored.
I87B · 96%
Try a prompt
Model card
From the Hugging Face model README
Model Card for Mistral-7B-Instruct-v0.1-8bit
The Mistral-7B-Instruct-v0.1-8bit is a 8bit quantize version with torch_dtype=torch.float16, I just load in 8bit and push here Mistral-7B-Instruct-v0.1.
For full details of this model please read our paper and release blog post.
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "mistralai/Mistral-7B-Instruct-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
load_in_8bit=True,
use_flash_attention_2=True,
torch_dtype=torch.float16,
)
model.push_to_hub("LsTam/Mistral-7B-Instruct-v0.1-8bit")
To use it:
from transformers import AutoTokenizer, AutoModelForCausalLM
tok_name = "mistralai/Mistral-7B-Instruct-v0.1"
model_name = "LsTam/Mistral-7B-Instruct-v0.1-8bit"
tokenizer = AutoTokenizer.from_pretrained(tok_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
use_flash_attention_2=True,
)
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- License
- apache-2.0
- Created
- Dec 14, 2023
- Updated
- Dec 14, 2023