Phi-4-mini-reasoning-GGUF

Phi-4-mini-reasoning GGUF

GGUF conversion and Q4_K_M quantization of Microsoft Phi-4-mini-reasoning for efficient local inference.

Original Model

Official model:

https://huggingface.co/microsoft/Phi-4-mini-reasoning

This repository contains community-created GGUF conversions of the original model.

Please refer to the official model card for complete information about the model, training, intended use, limitations, and license.

Files

Recommended Version

Q4_K_M

Phi-4-mini-reasoning-Q4_K_M.gguf is recommended for most local inference setups.

It provides a much smaller memory footprint than the F16 version while maintaining a good balance between quality and efficiency.

F16

The F16 version is provided for users who have sufficient memory and want to use a higher-precision GGUF representation.

Quantization

Conversion pipeline:

Microsoft Phi-4-mini-reasoning → F16 GGUF → Q4_K_M

The Q4_K_M file was generated using the quantization tools from llama.cpp.

Quantization:

Q4_K_M

No importance matrix was used.

llama.cpp

Basic inference:

llama-cli -m Phi-4-mini-reasoning-Q4_K_M.gguf

GPU offloading:

llama-cli -m Phi-4-mini-reasoning-Q4_K_M.gguf -ngl 99

Run a local server:

llama-server -m Phi-4-mini-reasoning-Q4_K_M.gguf -ngl 99

Compatible Software

The GGUF files can be used with software supporting the GGUF format, including:

• llama.cpp
• llama-cpp-python
• LM Studio
• Jan
• Other GGUF-compatible inference engines

Context

Phi-4-mini-reasoning is designed for reasoning-focused language generation.

For model-specific capabilities and limitations, refer to the official Microsoft model card.

License

The original model is provided under its respective Microsoft license.

Please review the official model repository before using or redistributing the model:

https://huggingface.co/microsoft/Phi-4-mini-reasoning

Disclaimer

This is a community GGUF conversion and quantization.

This repository is not an official Microsoft repository.

Conversion Environment

Conversion and quantization were performed using llama.cpp.

Conversion:

convert_hf_to_gguf.py

Quantization:

llama-quantize

Quantization type:

Q4_K_M