Downloads · 30 days
10
2% of all-time downloads
cloudyu_-_Pluto_13B_DPO-gguf
RichardErkhov/cloudyu_-_Pluto_13B_DPO-gguf
cloudyu_-_Pluto_13B_DPO-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
All-time downloads
468
Public
Repo size
108 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf123 GB · 100%
At a glance
- Access
- Public
- Created
- Nov 1, 2024
- Updated
- Nov 1, 2024
- SHA
- 854246a0
Model card
From the Hugging Face model README
Quantization made by Richard Erkhov.
Pluto_13B_DPO - GGUF
- Model creator: https://huggingface.co/cloudyu/
- Original model: https://huggingface.co/cloudyu/Pluto_13B_DPO/
| Name | Quant method | Size |
|---|---|---|
| Pluto_13B_DPO.Q2_K.gguf | Q2_K | 4.43GB |
| Pluto_13B_DPO.Q3_K_S.gguf | Q3_K_S | 5.2GB |
| Pluto_13B_DPO.Q3_K.gguf | Q3_K | 5.78GB |
| Pluto_13B_DPO.Q3_K_M.gguf | Q3_K_M | 5.78GB |
| Pluto_13B_DPO.Q3_K_L.gguf | Q3_K_L | 4.02GB |
| Pluto_13B_DPO.IQ4_XS.gguf | IQ4_XS | 4.92GB |
| Pluto_13B_DPO.Q4_0.gguf | Q4_0 | 6.78GB |
| Pluto_13B_DPO.IQ4_NL.gguf | IQ4_NL | 2.56GB |
| Pluto_13B_DPO.Q4_K_S.gguf | Q4_K_S | 3.66GB |
| Pluto_13B_DPO.Q4_K.gguf | Q4_K | 7.25GB |
| Pluto_13B_DPO.Q4_K_M.gguf | Q4_K_M | 3.43GB |
| Pluto_13B_DPO.Q4_1.gguf | Q4_1 | 0.34GB |
| Pluto_13B_DPO.Q5_0.gguf | Q5_0 | 7.79GB |
| Pluto_13B_DPO.Q5_K_S.gguf | Q5_K_S | 4.06GB |
| Pluto_13B_DPO.Q5_K.gguf | Q5_K | 8.51GB |
| Pluto_13B_DPO.Q5_K_M.gguf | Q5_K_M | 8.51GB |
| Pluto_13B_DPO.Q5_1.gguf | Q5_1 | 9.01GB |
| Pluto_13B_DPO.Q6_K.gguf | Q6_K | 9.84GB |
| Pluto_13B_DPO.Q8_0.gguf | Q8_0 | 12.75GB |
Original model description:
license: cc-by-nc-4.0 tags:
- moe
- dpo
DPO of cloudyu/Mixtral_7Bx2_MoE
dataset : jondurbin/truthy-dpo-v0.1
- metrics average 75.88
- TruthfulQA 81.5 SOTA (2024-01-17)
gpu code example
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
import math
## v2 models
model_path = "cloudyu/Mixtral_7Bx2_MoE_DPO"
tokenizer = AutoTokenizer.from_pretrained(model_path, use_default_system_prompt=False)
model = AutoModelForCausalLM.from_pretrained(
model_path, torch_dtype=torch.bfloat16, device_map='auto',local_files_only=False, load_in_4bit=True
)
print(model)
prompt = input("please input prompt:")
while len(prompt) > 0:
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to("cuda")
generation_output = model.generate(
input_ids=input_ids, max_new_tokens=500,repetition_penalty=1.2
)
print(tokenizer.decode(generation_output[0]))
prompt = input("please input prompt:")
- Created
- Nov 1, 2024
- Updated
- Nov 1, 2024