MiniCPM5-1B-GGUF

MiniCPM5-1B GGUF

GGUF quantizations of openbmb/MiniCPM5-1B, covering 30 files (18.1 GB total).

Files

Which one should I download?

Pick the largest file that leaves a couple of gigabytes of headroom on the device you will run it on — the model has to fit in RAM (or VRAM, if you are offloading) alongside the KV cache and the OS.

• Plenty of memory: Q6_K or Q8_0.
• The usual choice: Q4_K_M.
• Tight on memory: IQ4_XS, then IQ3_M, then IQ2_M.
• The IQ* files are imatrix-guided and generally beat a Q* file of similar size, at the cost of slightly slower inference on some hardware.

Quantization details

• Importance matrix computed with llama-imatrix over 500 rows of Salesforce/wikitext (wikitext-2-raw-v1).
• The matrix was computed on the BF16 weights.
• K-quants below 6 bit and the whole IQ set are imatrix-guided. Q4_0/Q4_1/Q5_0/Q5_1 are legacy round-to-nearest and ignore it; Q6_K/Q8_0 are near-lossless and do not need it.
• All files are cut from the same BF16 GGUF, so differences between them are quantization only.

Usage

llama-cli -hf NANI-Nithin/MiniCPM5-1B-GGUF:Q4_K_M -p "Hello"

Or download one file and point at it directly:

huggingface-cli download NANI-Nithin/MiniCPM5-1B-GGUF MiniCPM5-1B-Q4_K_M.gguf --local-dir .
llama-cli -m MiniCPM5-1B-Q4_K_M.gguf -p "Hello"

---

Quantized with llama.cpp by AgentQuantix on 2026-09-04. Licensing follows the base model.