Downloads · 30 days
13
22% of all-time downloads
Mistral-7B-Instruct-v0.1-8bit
dmitrybright/Mistral-7B-Instruct-v0.1-8bit
Mistral-7B-Instruct-v0.1-8bit is a text generation model from dmitrybright. Use it when you need the model to write or continue text. It is set up for transformers.
All-time downloads
60
Public
Parameters
7.2B
7.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors7.5 GB · 100%
Parameter types
How the weights are stored.
I87B · 96%
Try a prompt
Model card
From the Hugging Face model README
Mistral-7B-Instruct-v0.1-8bit
Create model
model_path = "mistralai/Mistral-7B-Instruct-v0.1"
bnb_config = BitsAndBytesConfig(
load_in_8bit=True
)
model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True, quantization_config=bnb_config, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_path)
Load in pipeline
text_generation_pipeline = transformers.pipeline(
model=model,
tokenizer=tokenizer,
task="text-generation",
eos_token_id=tokenizer.eos_token_id,
pad_token_id=tokenizer.eos_token_id,
repetition_penalty=1.1,
return_full_text=True,
max_new_tokens=100,
)
mistral_llm = HuggingFacePipeline(pipeline=text_generation_pipeline)
text = "what is mistral?"
mistral_llm.invoke(text)
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- Created
- Feb 9, 2024
- Updated
- Feb 9, 2024