Downloads · 30 days
60
20% of all-time downloads
vinci00/ministral-3-3b-gguf-q4
ministral-3-3b-gguf-q4 is a text generation model from vinci00. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
This repository contains a community GGUF conversion and Q4KM quantization of mistralai/Ministral-3-3B-Instruct-2512-BF16 for Ollama and llama.cpp-compatible runtimes.
Downloads · 30 days
60
20% of all-time downloads
All-time downloads
293
Public
Repo size
2.1 GB
Likes
0
Public
Click a slice to open those files.
.gguf2.1 GB · 100%
From the Hugging Face model README
This repository contains a community GGUF conversion and Q4_K_M
quantization of mistralai/Ministral-3-3B-Instruct-2512-BF16
for Ollama and llama.cpp-compatible runtimes.
This is a text-generation distribution. It is a separate GGUF quantization from the MLX 4-bit artifact; it was created from the same official BF16 checkpoint, not by converting MLX weights back to GGUF.
| File | Description |
|---|---|
Ministral-3-3B-Instruct-2512-Q4_K_M.gguf | 2.1 GB GGUF Q4_K_M model |
Modelfile | Ollama import recipe |
README.md | This model card |
Download the GGUF and Modelfile into the same directory, then run:
ollama create ministral-3-3b-mlx -f Modelfile
ollama run ministral-3-3b-mlx
The published model can also be referenced from the Hub using the Ollama Hugging Face integration when supported by the installed Ollama version:
ollama run hf.co/vinci00/ministral-3-3b-gguf-q4
The source checkpoint was the official Mistral AI BF16 Safetensors model. The
conversion used the current llama.cpp converter
and llama-quantize with Q4_K_M. The reproducible MLX conversion notebook,
benchmark, and source card are maintained at
vinci00/ministral-3-mlx-4bit
and in the course repository.
The same four fixed functional prompts used for the MLX artifact passed when the resulting model was loaded through Ollama on an Apple M4 Mac mini. This is a small smoke test, not a quality, safety, or task benchmark.
BF16 versus GGUF Q4_K_M accuracy has not been measured. The four smoke checks above do not estimate task accuracy or establish parity with BF16.
The source repository
now includes benchmark_accuracy.py: a paired GSM8K test evaluation for the
MLX unquantized and 4-bit checkpoints, with exact-match scoring, accuracy
changes in percentage points, uncertainty intervals, and chart output. Its
MLX inference backend does not accept this GGUF artifact or call Ollama.
A valid GGUF comparison still requires evaluating the original unquantized source and Q4_K_M on identical held-out questions, prompts, decoding settings, and scoring, preferably within the same runtime. Record both runtime and precision if comparing across runtimes. Do not copy MLX accuracy numbers into this card: the quantization formats and inference implementations differ. The MLX comparison measured 87/100 for both BF16 and MLX 4-bit on a fixed 100-example GSM8K test sample (2026-09-21, M1 Pro). This is evidence for the MLX artifacts only. No GSM8K accuracy result is claimed for this GGUF/Ollama distribution. See the source repository for raw results and methodology.
The upstream checkpoint is released under the Apache 2.0 license. Preserve the upstream attribution and model card when redistributing this derivative.