Downloads · 30 days
7
11% of all-time downloads
Zoyd/mlabonne_Llama-3-8B-Instruct-abliterated-dpomix-2_2bpw_exl2
mlabonne_Llama-3-8B-Instruct-abliterated-dpomix-2_2bpw_exl2 is a text generation model from Zoyd. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
Exllamav2 quant (exl2 / 2.2 bpw) made with ExLlamaV2 v0.1.1
Downloads · 30 days
7
11% of all-time downloads
All-time downloads
64
Public
Repo size
3.4 GB
Likes
0
Public
Click a slice to open those files.
.safetensors3.4 GB · 100%
From the Hugging Face model README
Exllamav2 quant (exl2 / 2.2 bpw) made with ExLlamaV2 v0.1.1
Other EXL2 quants:
| Quant | Model Size | lm_head |
|---|---|---|
| <center>2.2</center> | <center>3250 MB</center> | <center>6</center> |
| <center>2.5</center> | <center>3479 MB</center> | <center>6</center> |
| <center>3.0</center> | <center>3895 MB</center> | <center>6</center> |
| <center>3.5</center> | <center>4311 MB</center> | <center>6</center> |
| <center>3.75</center> | <center>4519 MB</center> | <center>6</center> |
| <center>4.0</center> | <center>4727 MB</center> | <center>6</center> |
| <center>4.25</center> | <center>4933 MB</center> | <center>6</center> |
| <center>5.0</center> | <center>5558 MB</center> | <center>6</center> |
| <center>6.0</center> | <center>6490 MB</center> | <center>8</center> |
| <center>6.5</center> | <center>6881 MB</center> | <center>8</center> |
| <center>8.0</center> | <center>8073 MB</center> | <center>8</center> |
This model is an experimental DPO fine-tune of an abliterated Llama 3 8B Instruct model on the full mlabonne/orpo-dpo-mix-40k dataset. It improves Llama 3 8B Instruct's performance while being uncensored.
| Model | Average | AGIEval | GPT4All | TruthfulQA | Bigbench |
|---|---|---|---|---|---|
| mlabonne/Llama-3-8B-Instruct-abliterated-dpomix 📄 | 52.26 | 41.6 | 69.95 | 54.22 | 43.26 |
| meta-llama/Meta-Llama-3-8B-Instruct 📄 | 51.34 | 41.22 | 69.86 | 51.65 | 42.64 |
| failspy/Meta-Llama-3-8B-Instruct-abliterated-v3 📄 | 51.21 | 40.23 | 69.5 | 52.44 | 42.69 |
| abacusai/Llama-3-Smaug-8B 📄 | 49.65 | 37.15 | 69.12 | 51.66 | 40.67 |
| mlabonne/OrpoLlama-3-8B 📄 | 48.63 | 34.17 | 70.59 | 52.39 | 37.36 |
| meta-llama/Meta-Llama-3-8B 📄 | 45.42 | 31.1 | 69.95 | 43.91 | 36.7 |
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "mlabonne/Llama-3-8B-Instruct-abliterated-dpomix"
messages = [{"role": "user", "content": "What is a large language model?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])