Mistral-7B-Instruct-v0.3-Q4_K_M

Mistral-7B-Instruct-v0.3-Q4_K_M (GGUF)

This repository contains the Mistral-7B-Instruct-v0.3 model in GGUF format with Q4_K_M quantization.

Model Information

• Base Model: Mistral-7B-Instruct-v0.3
• Original Creator: Mistral AI
• License: Apache 2.0
• Format: GGUF (Quantized)
• Quantization: Q4_K_M (4-bit quantization, medium quality)
• Model Size: ~4.1 GB

About This Model

Mistral 7B Instruct v0.3 is a instruction-tuned large language model developed by Mistral AI. This version has been quantized to GGUF format for efficient inference with llama.cpp and compatible frameworks.

Key Features

• 7 billion parameters
• Optimized for instruction-following tasks
• Supports extended vocabulary
• Apache 2.0 licensed (commercial use allowed)

Usage

This model can be used with:

• llama.cpp
• Ollama
• LM Studio
• text-generation-webui
• Any GGUF-compatible inference engine

Example with llama.cpp:

./main -m Mistral-7B-Instruct-v0.3-Q4_K_M.gguf -p "Your prompt here" -n 512

Example with Python (llama-cpp-python):

from llama_cpp import Llama

llm = Llama(model_path="Mistral-7B-Instruct-v0.3-Q4_K_M.gguf")
output = llm("Q: What is the capital of France? A:", max_tokens=256)
print(output)

Quantization Details

Q4_K_M quantization provides:

• Good balance between size and quality
• ~4-bit average quantization
• Suitable for most use cases
• Recommended for systems with limited VRAM

License

This model is licensed under Apache License 2.0.

You are free to:

• ✅ Use commercially
• ✅ Modify and distribute
• ✅ Use privately
• ✅ Patent use

Attribution Required: You must give appropriate credit to Mistral AI, provide a link to the license, and indicate if changes were made.

See the Apache 2.0 License for full details.

Attribution

Original model developed by Mistral AI.

• Original Model: mistralai/Mistral-7B-Instruct-v0.3
• GGUF Conversion: Quantized for efficient inference

Citation

@article{mistral7b,
  title={Mistral 7B},
  author={Mistral AI Team},
  year={2023}
}

Disclaimer

This model is provided "as is" without warranty of any kind. See the Apache 2.0 license for details.