minicpm5-2b-distilled-reasoning-gguf

MiniCPM5-2B Distilled Reasoning (GGUF)

Quantized GGUF version of jigs97022/minicpm5-2b-distilled-reasoning. Optimized for local CPU/GPU inference with llama.cpp, Ollama, LM Studio, and vLLM.

Model Origin

This model was distilled from three frontier models (Qwen3.8-Max, GLM-5.2, Kimi K3) onto MiniCPM5-2B using 10K high-quality reasoning traces. It excels at structured chain-of-thought problem solving in math, code, logic puzzles, and tool-use scenarios.

Quick Start

Ollama (Recommended)

ollama run hf.co/jigs97022/minicpm5-2b-distilled-reasoning-gguf

LM Studio

1. Search "jigs97022/minicpm5-2b-distilled-reasoning-gguf" in the search bar
2. Download the Q4_K_M quantization (or Q6_K / Q8_0 for higher quality)
3. Load and chat

llama.cpp / Python

from llama_cpp import Llama

llm = Llama.from_pretrained( repo_id="jigs97022/minicpm5-2b-distilled-reasoning-gguf", filename="*q4_k_m.gguf", n_ctx=4096, n_gpu_layers=-1 # Set to 0 for CPU-only )

output = llm.create_chat_completion( messages=[{"role": "user", "content": "If 3x + 7 = 22, what is x? Show step-by-step reasoning."}], max_tokens=512, temperature=0.7 ) print(output["choices"][0]["message"]["content"])

vLLM

vllm serve jigs97022/minicpm5-2b-distilled-reasoning-gguf
--max-model-len 4096
--gpu-memory-utilization 0.8

Quantization Details

Hardware Requirements

Chat Template

Uses native MiniCPM5 formatting. Do NOT override the chat template in llama.cpp/Ollama; the model expects its built-in template.

Limitations

• Context window limited to 4096 tokens
• Trained on 10K samples; may underperform full-corpus variant on rare edge cases
• Knowledge cutoff inherited from base MiniCPM5 (~2024)
• May produce verbose think sections; adjust temperature down to 0.5 if you prefer concise outputs

License

Apache-2.0 (inherited from base MiniCPM5-2B). Verify upstream distillation dataset license before commercial deployment.

Acknowledgments

• OpenBMB for MiniCPM5-2B
• r0b0tlab for the multi-teacher distillation dataset
• Unsloth for accelerated training and GGUF export
• llama.cpp team for GGUF format specification