Downloads · 30 days
92
1% of all-time downloads
olafgeibig/Nous-Hermes-2-Mistral-7B-DPO-GGUF
Nous-Hermes-2-Mistral-7B-DPO-GGUF is a machine learning model from olafgeibig. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
I converted NousResearch/Nous-Hermes-2-Mistral-7B-DPO to GGUF and quantized it to my favorite quantizations. Se their original model card for all the details.
Downloads · 30 days
92
1% of all-time downloads
All-time downloads
6.3K
Public
Repo size
27.8 GB
Likes
0
Public
Click a slice to open those files.
.gguf22.9 GB · 100%
From the Hugging Face model README
I converted NousResearch/Nous-Hermes-2-Mistral-7B-DPO to GGUF and quantized it to my favorite quantizations. Se their original model card for all the details.
I quickly quantized this model using a modified version of AutoGGUF from Maxime Labonne
Here is my Ollama modelfile. According to llama.cpp the model was trained on 32k tokens, but I set the ollama modelfile to 16k so that 16 GB Macs can still run it.
FROM ./nous-hermes-2-mistral-7b-dpo.Q5_K_M.gguf
PARAMETER num_ctx 16384
TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"