MiniCPM5-1B-GGUF

MiniCPM5-1B (GGUF Quantizations)

This repository contains custom GGUF format quantizations of the openbmb/MiniCPM5-1B model.

MiniCPM5-1B is a highly capable 1-billion parameter Transformer built for on-device, local deployment, and resource-constrained scenarios. It utilizes a standard LlamaForCausalLM architecture, features hybrid reasoning (built-in <think> tokens), and supports a massive 131k context window.

📦 Available Files and Quantizations

These models were quantized specifically for high-efficiency CPU/Edge inference using the llama.cpp framework.

🚀 Quick Start with llama.cpp

Because MiniCPM5-1B uses standard Llama architecture, it is fully supported by llama.cpp out of the box. No custom forks or kernels are required.

1. Interactive CLI

To run the model directly in your terminal using CPU threads:

./llama-cli -m minicpm5-1b-Q4_K_M.gguf -p "Artificial intelligence and local model deployment are transforming technology because" -n 256 -t 4