Ministral-3-8B-Reasoning-2512-GGUF

Ministral-3-8B-Reasoning-2512

Ministral-3-8B-Reasoning-2512 is an efficient multimodal reasoning-focused language model developed by Mistral AI, optimized for structured reasoning, coding workflows, conversational AI, and vision-language tasks. This repository contains GGUF quantized variants of the model optimized for efficient local inference using llama.cpp.

The model combines a compact language model with a vision encoder, enabling both text and image understanding capabilities while remaining practical for edge and consumer-grade deployments. The quantized formats significantly reduce memory requirements while preserving strong reasoning and instruction-following performance.

---

Model Overview

• Model Name: Ministral-3-8B-Reasoning-2512
• Base Model: mistralai/Ministral-3-8B-Reasoning-2512
• Architecture: Decoder-only Transformer with Vision Encoder
• Parameter Count: ~8.4B Language Model + ~0.4B Vision Encoder
• Context Window: 256K tokens
• Modalities: Text, Image (multimodal support)
• Primary Languages: Multilingual
• Developer: Mistral AI
• License: Apache 2.0

---

Quantization Formats

This repository provides various GGUF quantized versions of the Ministral-3-8B-Reasoning-2512 model, optimized for efficient local inference using llama.cpp. Below are the details of the available I-Matrix (IQ) formats.

IQ3_M

• Size reduction of approx 76.49% (3.72 GB) compared to 16-bit (15.82 GB)
• Aggressive 3-bit quantization optimized for maximum memory efficiency
• Suitable for CPU-only inference and lightweight deployment environments
• Enables practical deployment on low-memory consumer hardware and edge systems
• Output quality may reduce on highly complex reasoning, coding, and multimodal analytical tasks

IQ4_NL

• Size reduction of approx 70.73% (4.63 GB) compared to 16-bit (15.82 GB)
• Advanced 4-bit non-linear quantization designed to better preserve reasoning quality and structured outputs
• More suitable for coding workflows, analytical reasoning, and multimodal understanding tasks
• Typically provides stronger consistency compared to lower-bit formats
• Slightly increased computational overhead during inference

IQ4_XS

• Size reduction of approx 72.12% (4.41 GB) compared to 16-bit (15.82 GB)
• Balanced 4-bit quantization focused on efficient inference and stable reasoning performance
• Good trade-off between model size, response quality, and inference speed
• Suitable for conversational AI, reasoning workflows, and structured generation tasks
• Maintains reliable performance across most practical instruction-following and multimodal workloads

---

Training Background (Original Model)

Ministral-3-8B-Reasoning-2512 is trained with an emphasis on efficient reasoning performance, multimodal understanding, structured response generation, and reliable instruction following across a variety of downstream tasks.

Pretraining

• Large-scale language and multimodal pretraining across diverse datasets
• Focus on contextual understanding, image understanding, and efficient reasoning capability
• Optimized for downstream conversational, analytical, and multimodal workloads

Instruction Tuning

• Refined using instruction-following and reasoning-oriented datasets
• Enhanced for structured responses and multi-step reasoning tasks
• Improved consistency for coding, analytical, conversational, and vision-language workflows

---

Key Capabilities

• Multimodal Understanding Supports image and text understanding for vision-language reasoning workflows.

• Reasoning and Analysis Performs effectively on structured reasoning and multi-step analytical tasks.

• Instruction Following Produces context-aware and structured responses across diverse prompts.

• Coding Assistance Supports code explanation, generation, and technical reasoning workflows.

• Efficient Edge Deployment Compact architecture enables practical local inference across constrained hardware environments.

• Large Context Processing Supports extended-context reasoning and long conversational interactions through a 256K context window.

---

Usage Example

Using llama.cpp

./llama-cli \
  -m SandlogicTechnologies/Ministral-3-8B-Reasoning-2512_IQ4_NL.gguf \
  -p "Analyze the attached image and explain the mathematical diagram step-by-step."

---

Recommended Usecases

• Multimodal AI Assistants Build lightweight local assistants capable of handling text and image inputs.

• Reasoning and Analytical Tasks Generate structured outputs for logical and multi-step problem-solving workflows.

• Coding and Technical Assistance Support code explanation, debugging, and development-oriented workflows.

• Vision-Language Applications Perform image understanding and multimodal reasoning tasks.

• Research and Experimentation Evaluate prompting strategies, multimodal workflows, and local inference pipelines.

---

Acknowledgments

These quantized models are based on the original work by the Mistral AI development team.

Special thanks to:

• The Mistral AI team for developing and releasing the Ministral-3-8B-Reasoning-2512 model.

• Georgi Gerganov and the llama.cpp open-source community for enabling efficient quantization and inference via the GGUF format.

---

Contact

For questions, feedback, or support, please reach out at [email protected] or visit https://www.sandlogic.com/