Downloads · 30 days
2
4% of all-time downloads
mpasila/Llama3-8B-OpenHermes-DPO-exl2-4bpw
Llama3-8B-OpenHermes-DPO-exl2-4bpw is a text generation model from mpasila. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.
This is an ExLlamaV2 quantized model in 4bpw of Muhammad2003/Llama3-8B-OpenHermes-DPO using the default calibration dataset.
Downloads · 30 days
2
4% of all-time downloads
All-time downloads
49
Public
Repo size
5 GB
Likes
0
Public
Click a slice to open those files.
.safetensors5 GB · 100%
From the Hugging Face model README
This is an ExLlamaV2 quantized model in 4bpw of Muhammad2003/Llama3-8B-OpenHermes-DPO using the default calibration dataset.

Llama3-8B-OpenHermes-DPO is DPO-Finetuned model of Llama3-8B, on the OpenHermes-2.5 preference dataset using QLoRA.
</details><br>!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "Muhammad2003/Llama3-8B-OpenHermes-DPO"
messages = [{"role": "user", "content": "What is a large language model?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])