Phi-3-mini-4k-instruct-GGUF

Phi 3 Mini 4K Instruct GGUF

Updated with Microsoft’s latest model changes as of July 21, 2024

Original model: Phi-3-mini-4k-instruct

Model creator: Microsoft

This repo contains GGUF format model files for Microsoft’s Phi 3 Mini 4K Instruct.

The Phi-3-Mini-4K-Instruct is a 3.8B parameters, lightweight, state-of-the-art open model trained with the Phi-3 datasets that includes both synthetic data and the filtered publicly available websites data with a focus on high-quality and reasoning dense properties.

Learn more on Microsoft’s Model page.

What is GGUF?

GGUF is a file format for representing AI models. It is the third version of the format, introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. Converted with llama.cpp build 3432 (revision 45f2c19), using autogguf.

Prompt template

<|system|>
{{system_prompt}}<|end|>
<|user|>
{{prompt}}<|end|>
<|assistant|>

---

Download & run with cnvrs on iPhone, iPad, and Mac!

cnvrs is the best app for private, local AI on your device:

• create & save Characters with custom system prompts & temperature settings
• download and experiment with any GGUF model you can find on HuggingFace!
• make it your own with custom Theme colors
• powered by Metal ⚡️ & Llama.cpp, with haptics during response streaming!
• try it out yourself today, on Testflight!
• follow cnvrs on twitter to stay up to date

---

Original Model Evaluation

Comparison of July update vs original April release:

---

Original April release

As is now standard, we use few-shot prompts to evaluate the models, at temperature 0. The prompts and number of shots are part of a Microsoft internal tool to evaluate language models, and in particular we did no optimization to the pipeline for Phi-3. More specifically, we do not change prompts, pick different few-shot examples, change prompt format, or do any other form of optimization for the model.

The number of k–shot examples is listed per-benchmark.