Magistral-Small-2509-Vision-GGUF

<img src="https://huggingface.co/EnlistedGhost/Magistral-Small-2509-Vision-GGUF/resolve/main/resources/Magistral_Icon_MistralAI.png" alt="Magistral image" width="395" height="752">

-----------------------------------------------<br /> - Model Details and Specifications: -<br />-----------------------------------------------

-----------------------------------------------<br /> - Update July 11th 2026 -<br />-----------------------------------------------

• Fixed MMPROJ Multi-Modal Vision projector. (Re-Uploaded with working projector)
• Uploaded brand new Q2_K_M, Q3_K_M, Q4_K_M, Q5_K_XL, Q6_K_L quants, using the GH5TS method! These offer significantly higher performance and quality of response over previously uploaded release files with the same or similar size of file!
• All files are being re-Uploaded with brand new and custom Quantize formulae: Over the next couple days the files will be updated with the new "GH5TS" (Pronounced: Ghosts) Quant method! This quantization method is a dynamic quantizaton that bases all quants no matter the size to include Q5_K in layers that are critical to cognitive abilities of the model while allowing for lower Quantization bit values for non-crucial layers.
• New chat template! Yes, a new chat template that completely fixes previous issues for llama.cpp users and retains even higher performance than the originally made template for Ollama/Xllama users as well!

<br /><br />

------------------------------------------------------------

Magistral-Small-2509-Vision (24B Parameters)

------------------------------------------------------------

Description: <br /> This model was re-configured with MistralAI's original Magistral-Small vision tower, using only official MistralAI files, weights and model data. This has resulted in a Vision Multimodal version of Magistral-Small-2509 including fully functional Vision (re-enabled). <br /><br /> The Chat-template and System-prompt have been reworked and customized to improve performance and quality across all Quantizated Files. No modifications, edits, or additional configurations are required to use this model with Ollama/Llama.cpp Both Vision and Text work. (^.^) <br />

This release contains: <br /> Llama.cpp, Ollama, and Xllama compatible GGUF converted and Quantized model files <br /> (Compatible with Xllama, Ollama, and Llama.cpp) <br />

IMPORTANT NOTICE as of (GMT-8) 07:00 July 11th 2026: <br /> Please note: The chat-template AND the system-prompt have been rewritten and finalized but differ from what MistralAI made available. You can still use MistralAI's default template and prompt, however it is recommended to be using what is provided within this release. Below is a copy of the new chat template (Default System Message Removed for Ease of Reading).

Chat Template:

{%- set ns = namespace(remMessage=false, hasSys=false, injSystem=true) -%}
{%- for msg in messages -%}
    {%- if msg.role == "system" -%}
        {%- set ns.hasSys = true -%}
    {%- endif -%}
{%- endfor -%}
{{- bos_token }}
{%- for msg in messages -%}
    {%- if ns.injSystem -%}
        [SYSTEM_PROMPT]
        {%- if ns.hasSys -%}
            {{ msg.content }}
        {%- else -%}
            'Default System Message Is provided in the actual GGUF file here'
        {%- endif -%}
        [/SYSTEM_PROMPT]
        {%- set ns.injSystem = false -%}
    {%- endif -%}
    {%- if (messages|length - loop.index0) < 29 -%}
        {%- set ns.remMessage = true -%}
    {%- endif -%}
    {%- if ns.remMessage -%}
        {%- if msg.role == "user" -%}
            [INST]
            {%- if msg.content is string %}
                {{ msg.content }}
            {%- else %}
                {%- for block in msg.content %}
                    {%- if block.type == 'text' %}
                        {{- block.text }}
                    {%- if block.type in ['image', 'image_url'] %}
                        [IMG]
                    {%- endif %}
                {%- endfor %}
            {%- endif %}
            [/INST]
        {%- elif msg.role == "assistant" -%}
            {{ msg.content }}
        {%- endif -%}
    {%- endif -%}
    {{- eos_token }}
{%- endfor -%}

<br /><br />

Happy LLM Inferrencing,<br /> -- Jon Z (EnlistedGhost)

---

---------------------------------------------------<br /> - Conversion and GGUF Quantization: -<br />---------------------------------------------------

Quantized GGUF version of:

• EnlistedGhost/Magistral-Small-2509-Vision <br /> (by MistralAI - modified by EnlistedGhost)

Original Model Link (Safetensors):

• EnlistedGhost/Magistral-Small-2509-Vision

Converted using: Llama.cpp (b9840) Quantized using: Llama.cpp (b9760)

---

--------------------------------------<br /> ---- How to run this Model ---- <br /> --------------------------------------

Compatible Software (Required to use this Model) <br /> You can run this model by using either Ollama (or) Llama.cpp <br /> (Below are instruction on running these GGUF files with Ollama)

How to run this Model using Ollama <br /> You can run this model by using the "ollama run" command.<br /> Simply copy & paste one of the commands from the list below into<br /> your console, terminal or power-shell window.

Vision Projector (Files) <br /> mmproj (Vision Projector) Files

---

-------------------------------------------------<br /> - Legal, Citations and Usage Details: -<br />-------------------------------------------------

Intended Use

Same as original:

• [MistralAI]

Out-of-Scope Use

Same as original:

• [MistralAI]

Bias, Risks, and Limitations

Same as original:

• [MistralAI]

Evaluation

• This model has NOT been evaluated in any form, scope or method of use.
• !!! USE AT YOUR OWN RISK !!!
• !!! NO WARRANTY IS PROVIDED OF ANY KIND !!!

---

Citation (Original Paper)

[MistalAI Magistral-Small-2509 Original Paper]

Detailed Release Information

• Originally Developed by: [MistralAI]
• Modified with Vision re-Enabled by: [EnlistedGhost]
• MMPROJ (Vision) Quantized by: [EnlistedGhost]
• Model Quantized for GGUF by: [EnlistedGhost]
• Model type & format: [Quantized/GGUF]
• License type: [Apache-2.0]

Model Card Authors and Contact

[EnlistedGhost]