Phi-4-mini-instruct-Q4_K_M-GGUF

Phi-4-mini-instruct Q4_K_M GGUF

This repository contains a GGUF Q4_K_M quantized version of Microsoft's Phi-4-mini-instruct model.

Q4_K_M Quantized version can be downloaded here

The model was converted to the GGUF format and quantized using llama.cpp for local inference as part of my AI-Powered Learning Assistant project.

Base Model

The original model is Microsoft Phi-4-mini-instruct:

microsoft/Phi-4-mini-instruct

Original model: https://huggingface.co/microsoft/Phi-4-mini-instruct

This repository does not contain a model trained from scratch. It contains a converted and quantized version of the original Phi-4-mini-instruct model.

Quantization

• Format: GGUF
• Quantization: Q4_K_M
• Base model: Phi-4-mini-instruct
• Parameters: 3.8B

Using llama.cpp (https://github.com/ggml-org/llama.cpp) for quantization.

The Q4_K_M quantization was selected after testing different quantized versions of the model for local summarization, flashcard generation, and quiz generation.

It provided a suitable balance between generation quality, inference performance, and memory usage on the development hardware.

Intended Use

This quantized model was prepared for local inference in an AI-Powered Learning Assistant desktop application.

Within the project, the model is used for:

• Summarization of study material
• Flashcard generation
• Quiz generation

The model is executed locally using llama.cpp-compatible tooling.

Acknowledgements

The original Phi-4-mini-instruct model was developed by Microsoft.

Please refer to the original model repository and model card for model details, limitations, responsible AI considerations, and licensing information.

License

The base Phi-4-mini-instruct model is provided under the MIT License. Refer to the original Microsoft model repository for the complete license and usage information.