Downloads · 30 days
213
7% of all-time downloads
RichardErkhov/rasyosef_-_Phi-1_5-Instruct-v0.1-gguf
rasyosef_-_Phi-1_5-Instruct-v0.1-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
213
7% of all-time downloads
All-time downloads
3.2K
Public
Repo size
16.6 GB
Likes
0
Public
Click a slice to open those files.
.gguf19.2 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
Phi-1_5-Instruct-v0.1 - GGUF
| Name | Quant method | Size |
|---|---|---|
| Phi-1_5-Instruct-v0.1.Q2_K.gguf | Q2_K | 0.54GB |
| Phi-1_5-Instruct-v0.1.IQ3_XS.gguf | IQ3_XS | 0.59GB |
| Phi-1_5-Instruct-v0.1.IQ3_S.gguf | IQ3_S | 0.61GB |
| Phi-1_5-Instruct-v0.1.Q3_K_S.gguf | Q3_K_S | 0.61GB |
| Phi-1_5-Instruct-v0.1.IQ3_M.gguf | IQ3_M | 0.64GB |
| Phi-1_5-Instruct-v0.1.Q3_K.gguf | Q3_K | 0.68GB |
| Phi-1_5-Instruct-v0.1.Q3_K_M.gguf | Q3_K_M | 0.68GB |
| Phi-1_5-Instruct-v0.1.Q3_K_L.gguf | Q3_K_L | 0.75GB |
| Phi-1_5-Instruct-v0.1.IQ4_XS.gguf | IQ4_XS | 0.74GB |
| Phi-1_5-Instruct-v0.1.Q4_0.gguf | Q4_0 | 0.77GB |
| Phi-1_5-Instruct-v0.1.IQ4_NL.gguf | IQ4_NL | 0.78GB |
| Phi-1_5-Instruct-v0.1.Q4_K_S.gguf | Q4_K_S | 0.78GB |
| Phi-1_5-Instruct-v0.1.Q4_K.gguf | Q4_K | 0.83GB |
| Phi-1_5-Instruct-v0.1.Q4_K_M.gguf | Q4_K_M | 0.83GB |
| Phi-1_5-Instruct-v0.1.Q4_1.gguf | Q4_1 | 0.85GB |
| Phi-1_5-Instruct-v0.1.Q5_0.gguf | Q5_0 | 0.92GB |
| Phi-1_5-Instruct-v0.1.Q5_K_S.gguf | Q5_K_S | 0.92GB |
| Phi-1_5-Instruct-v0.1.Q5_K.gguf | Q5_K | 0.95GB |
| Phi-1_5-Instruct-v0.1.Q5_K_M.gguf | Q5_K_M | 0.95GB |
| Phi-1_5-Instruct-v0.1.Q5_1.gguf | Q5_1 | 1.0GB |
| Phi-1_5-Instruct-v0.1.Q6_K.gguf | Q6_K | 1.08GB |
| Phi-1_5-Instruct-v0.1.Q8_0.gguf | Q8_0 | 1.4GB |
library_name: transformers license: mit base_model: microsoft/phi-1_5 datasets:
The language model phi-1.5 is a Transformer with 1.3 billion parameters. It was trained using the same data sources as phi-1, augmented with a new data source that consists of various NLP synthetic texts. When assessed against benchmarks testing common sense, language understanding, and logical reasoning, phi-1.5 demonstrates a nearly state-of-the-art performance among models with less than 10 billion parameters.
The model has underwent a post-training process that incorporates both supervised fine-tuning and direct preference optimization for instruction following. I used the trl library and a single A100 40GB GPU during both the SFT and DPO steps.
Supervised Fine-Tuning
Direct Preference Optimization (DPO)
Given the nature of the training data, the Phi-1.5 Instruct model is best suited for prompts using the chat format as follows. You can provide the prompt as a question with a generic template as follows:
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Question?<|im_end|>
<|im_start|>assistant
For example:
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
How to explain Internet for a medieval knight?<|im_end|>
<|im_start|>assistant
where the model generates the text after <|im_start|>assistant .
This code snippets show how to get quickly started with running the model on a GPU:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
torch.random.manual_seed(0)
model_id = "rasyosef/Phi-1_5-Instruct-v0.1"
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="cuda",
torch_dtype="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
messages = [
{"role": "system", "content": "You are a helpful AI assistant."},
{"role": "user", "content": "Can you provide ways to eat combinations of bananas and dragonfruits?"},
{"role": "assistant", "content": "Sure! Here are some ways to eat bananas and dragonfruits together: 1. Banana and dragonfruit smoothie: Blend bananas and dragonfruits together with some milk and honey. 2. Banana and dragonfruit salad: Mix sliced bananas and dragonfruits together with some lemon juice and honey."},
{"role": "user", "content": "What about solving an 2x + 3 = 7 equation?"},
]
pipe = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
)
generation_args = {
"max_new_tokens": 500,
"return_full_text": False,
"temperature": 0.0,
"do_sample": False,
}
output = pipe(messages, **generation_args)
print(output[0]['generated_text'])
Note: If you want to use flash attention, call AutoModelForCausalLM.from_pretrained() with attn_implementation="flash_attention_2"
This model outperforms HuggingFace's SmolLM-1.7B-Instruct and the TinyLlama-1.1B-Chat-v1.0 models on all 5 of the following benchmarks. These benchmarks were run using EleutherAI's lm-evaluation-harness
| Model | Size (# params) | IFEval | GSM8K | MMLU | TruthfulQA | Winogrande |
|---|---|---|---|---|---|---|
| Phi-1_5-Instruct-v0.1 | 1.4B | 26.71 | 41.78 | 39.72 | 47.9 | 70.4 |
| SmolLM-1.7B-Instruct | 1.7B | 24.21 | 3.45 | 23.57 | 47.38 | 63.61 |
| TinyLlama-1.1B-Chat-v1.0 | 1.1B | 21.23 | 0 | 24.03 | 39.14 | 61.01 |
| phi-1_5 | 1.4B | 20.51 | 31.73 | 42.48 | 40.86 | 71.74 |