Downloads · 30 days
16
2% of all-time downloads
nisten/mistral-instruct0.2-imatrix4bit.gguf
mistral-instruct0.2-imatrix4bit.gguf is a machine learning model from nisten. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Base model: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2
Downloads · 30 days
16
2% of all-time downloads
All-time downloads
986
Public
Repo size
18.8 GB
Likes
5
Public
Click a slice to open those files.
.gguf18.8 GB · 100%
From the Hugging Face model README
Base model: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2
This is just a custom 4bit imatrix quant made to run optiomally on a macbook with 8gb of ram.
For use with llama.cpp https://github.com/ggerganov/llama.cpp