Mistral-Small-3.1-24B-Instruct-2503-GGUF

<img src="https://huggingface.co/api/resolve-cache/models/EnlistedGhost/Mistral-Small-3.1-24B-Instruct-2503-GGUF/1b5df8518e9317b8e82c606331f98eac7e6b4599/images%2FMistral_Small_3_1.jpeg?%2FEnlistedGhost%2FMistral-Small-3.1-24B-Instruct-2503-GGUF%2Fresolve%2Fmain%2Fimages%2FMistral_Small_3_1.jpeg=&etag=%22e8010bcf5402b9c637d49a480062861f083094be-inline%22" alt="Loading..." width="241" height="192">

-----------------------------------------------<br /> - Model Details and Specifications: -<br />-----------------------------------------------

Mistral Small 3.1 24B Instruct 2503 GGUF (Ollama & Llama.cpp)

This release contains: <br /> Llama.cpp and Ollama compatible GGUF converted and Quantized model files (Compatible with both Ollama, and Llama.cpp) <br /> (More information and updates just landed! More on the way (^.^) - Thank you for taking the time to view this release!)

Quantized GGUF version of:

• mistralai/Mistral-Small-3.1-24B-Instruct-2503 <br /> (by MistralAI)

Original Model Link:

• mistralai/Mistral-Small-3.1-24B-Instruct-2503

---

---------------------------------------------------<br /> - Conversion and GGUF Quantization: -<br />---------------------------------------------------

Software used to convert Safetensors to GGUF:

• <a href="https://github.com/ggml-org/llama.cpp/">llama.cpp, release verson: (b7540)</a>

Software used to create Quantized GGUF Files:

• <a href="https://github.com/ggml-org/llama.cpp/">llama.cpp, release verson: (b7540)</a>

Specific GitHub Commit Point:

• <a href="https://github.com/ggml-org/llama.cpp/commit/85c40c9b02941ebf1add1469af75f1796d513ef4">b7540</a>

Converted to GGUF and Quantized by:

• EnlistedGhost

---

-------------------------------<br /> ---- Updates & News ---- <br /> -------------------------------

Model Updates (as of: January 25th, 2026)

• Uploaded: Remaining GGUF Converted and Quantized model files
• Updated: ModelCard <br /> (this page)
• Planned Updates: Yes, more updates coming! Thank you whoever you are for your patience (^.^)

---

--------------------------------------<br /> ---- How to run this Model ---- <br /> --------------------------------------

Compatible Software (Required to use this Model) <br /> You can run this model by using either Ollama (or) Llama.cpp <br /> (Below are instruction on running these GGUF files with Ollama)

How to run this Model using Ollama <br /> You can run this model by using the "ollama run" command.<br /> Simply copy & paste one of the commands from the list below into<br /> your console, terminal or power-shell window.

Vision Projector (Files) <br /> mmproj (Vision Projector) Files

---

---------------------------<br /> ---- Original Info ---- <br /> ---------------------------

(Crossposted from the link in the above section: "Model Details"): <br /> <br />

Building upon Mistral Small 3 (2501), Mistral Small 3.1 (2503) adds state-of-the-art vision understanding and enhances long context capabilities up to 128k tokens without compromising text performance. With 24 billion parameters, this model achieves top-tier capabilities in both text and vision tasks.
This model is an instruction-finetuned version of: Mistral-Small-3.1-24B-Base-2503.

Mistral Small 3.1 can be deployed locally and is exceptionally "knowledge-dense," fitting within a single RTX 4090 or a 32GB RAM MacBook once quantized.

It is ideal for:

• Fast-response conversational agents.
• Low-latency function calling.
• Subject matter experts via fine-tuning.
• Local inference for hobbyists and organizations handling sensitive data.
• Programming and math reasoning.
• Long document understanding.
• Visual understanding.

For enterprises requiring specialized capabilities (increased context, specific modalities, domain-specific knowledge, etc.), we will release commercial models beyond what Mistral AI contributes to the community.

Learn more about Mistral Small 3.1 in our blog post.

Key Features

• Vision: Vision capabilities enable the model to analyze images and provide insights based on visual content in addition to text.
• Multilingual: Supports dozens of languages, including English, French, German, Greek, Hindi, Indonesian, Italian, Japanese, Korean, Malay, Nepali, Polish, Portuguese, Romanian, Russian, Serbian, Spanish, Swedish, Turkish, Ukrainian, Vietnamese, Arabic, Bengali, Chinese, Farsi.
• Agent-Centric: Offers best-in-class agentic capabilities with native function calling and JSON outputting.
• Advanced Reasoning: State-of-the-art conversational and reasoning capabilities.
• Apache 2.0 License: Open license allowing usage and modification for both commercial and non-commercial purposes.
• Context Window: A 128k context window.
• System Prompt: Maintains strong adherence and support for system prompts.
• Tokenizer: Utilizes a Tekken tokenizer with a 131k vocabulary size.

Benchmark Results

When available, we report numbers previously published by other model providers, otherwise we re-evaluate them using our own evaluation harness.

Pretrain Evals

Instruction Evals

Text

Vision

Multilingual Evals

Long Context Evals