Downloads · 30 days
19
2% of all-time downloads
Mistral-7B-Instruct-v0.1-AWQ
MaziyarPanahi/Mistral-7B-Instruct-v0.1-AWQ
Mistral-7B-Instruct-v0.1-AWQ is a text generation model from MaziyarPanahi. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
MaziyarPanahi/Mistral-7B-Instruct-v0.1-AWQ is a quantized (AWQ) version of mistralai/Mistral-7B-Instruct-v0.1
All-time downloads
821
Public
Parameters
7.2B
4.2 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors4.2 GB · 100%
Parameter types
How the weights are stored.
I327B · 96%
Try a prompt
Base models
Model card
From the Hugging Face model README
Description
MaziyarPanahi/Mistral-7B-Instruct-v0.1-AWQ is a quantized (AWQ) version of mistralai/Mistral-7B-Instruct-v0.1
How to use
Install the necessary packages
pip install --upgrade accelerate autoawq transformers
Example Python code
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "MaziyarPanahi/Mistral-7B-Instruct-v0.1-AWQ"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id).to(0)
text = "User:\nHello can you provide me with top-3 cool places to visit in Paris?\n\nAssistant:\n"
inputs = tokenizer(text, return_tensors="pt").to(0)
out = model.generate(**inputs, max_new_tokens=300)
print(tokenizer.decode(out[0], skip_special_tokens=True))
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- License
- apache-2.0
- Created
- Feb 9, 2024
- Updated
- Feb 9, 2024