Downloads · 30 days
110
6% of all-time downloads
RichardErkhov/miguelcarv_-_phi-1_5-slimorca-gguf
miguelcarv_-_phi-1_5-slimorca-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
110
6% of all-time downloads
All-time downloads
1.7K
Public
Repo size
14.6 GB
Likes
0
Public
Click a slice to open those files.
.gguf17.3 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
phi-1_5-slimorca - GGUF
| Name | Quant method | Size |
|---|---|---|
| phi-1_5-slimorca.Q2_K.gguf | Q2_K | 0.54GB |
| phi-1_5-slimorca.Q3_K_S.gguf | Q3_K_S | 0.61GB |
| phi-1_5-slimorca.Q3_K.gguf | Q3_K | 0.69GB |
| phi-1_5-slimorca.Q3_K_M.gguf | Q3_K_M | 0.69GB |
| phi-1_5-slimorca.Q3_K_L.gguf | Q3_K_L | 0.75GB |
| phi-1_5-slimorca.IQ4_XS.gguf | IQ4_XS | 0.74GB |
| phi-1_5-slimorca.Q4_0.gguf | Q4_0 | 0.77GB |
| phi-1_5-slimorca.IQ4_NL.gguf | IQ4_NL | 0.78GB |
| phi-1_5-slimorca.Q4_K_S.gguf | Q4_K_S | 0.78GB |
| phi-1_5-slimorca.Q4_K.gguf | Q4_K | 0.83GB |
| phi-1_5-slimorca.Q4_K_M.gguf | Q4_K_M | 0.83GB |
| phi-1_5-slimorca.Q4_1.gguf | Q4_1 | 0.85GB |
| phi-1_5-slimorca.Q5_0.gguf | Q5_0 | 0.92GB |
| phi-1_5-slimorca.Q5_K_S.gguf | Q5_K_S | 0.92GB |
| phi-1_5-slimorca.Q5_K.gguf | Q5_K | 0.96GB |
| phi-1_5-slimorca.Q5_K_M.gguf | Q5_K_M | 0.96GB |
| phi-1_5-slimorca.Q5_1.gguf | Q5_1 | 1.0GB |
| phi-1_5-slimorca.Q6_K.gguf | Q6_K | 1.09GB |
| phi-1_5-slimorca.Q8_0.gguf | Q8_0 | 1.41GB |
Phi 1.5 finetuned on SlimOrca-Dedup. This model was trained with the goal of giving Phi 1.5 the ablity to generate the EOS token together with being capable of doing beam search. It can also follow custom system prompts as shown in the example below.
import torch
import transformers
model = transformers.AutoModelForCausalLM.from_pretrained(
"miguelcarv/phi-1_5-slimorca",
trust_remote_code=True
)
tokenizer = transformers.AutoTokenizer.from_pretrained("microsoft/phi-1_5")
SYSTEM_PROMPT = "You are an AI assistant. You will be given a task. You must generate a detailed and long answer."
input_text = f"""{SYSTEM_PROMPT}
Instruction: Give me the first 5 prime numbers and explain what prime numbers are.
Output:"""
with torch.no_grad():
outputs = model.generate(
tokenizer(input_text, return_tensors="pt")['input_ids'],
max_length=1024,
num_beams = 3,
eos_token_id = tokenizer.eos_token_id
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))