Downloads · 30 days
229
9% of all-time downloads
RichardErkhov/RaushanTurganbay_-_GPT2_sft_and_dpo_tuned-gguf
RaushanTurganbay_-_GPT2_sft_and_dpo_tuned-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
229
9% of all-time downloads
All-time downloads
2.6K
Public
Repo size
4.4 GB
Likes
0
Public
Click a slice to open those files.
.gguf5.2 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
GPT2_sft_and_dpo_tuned - GGUF
| Name | Quant method | Size |
|---|---|---|
| GPT2_sft_and_dpo_tuned.Q2_K.gguf | Q2_K | 0.17GB |
| GPT2_sft_and_dpo_tuned.Q3_K_S.gguf | Q3_K_S | 0.19GB |
| GPT2_sft_and_dpo_tuned.Q3_K.gguf | Q3_K | 0.21GB |
| GPT2_sft_and_dpo_tuned.Q3_K_M.gguf | Q3_K_M | 0.21GB |
| GPT2_sft_and_dpo_tuned.Q3_K_L.gguf | Q3_K_L | 0.23GB |
| GPT2_sft_and_dpo_tuned.IQ4_XS.gguf | IQ4_XS | 0.22GB |
| GPT2_sft_and_dpo_tuned.Q4_0.gguf | Q4_0 | 0.23GB |
| GPT2_sft_and_dpo_tuned.IQ4_NL.gguf | IQ4_NL | 0.23GB |
| GPT2_sft_and_dpo_tuned.Q4_K_S.gguf | Q4_K_S | 0.23GB |
| GPT2_sft_and_dpo_tuned.Q4_K.gguf | Q4_K | 0.25GB |
| GPT2_sft_and_dpo_tuned.Q4_K_M.gguf | Q4_K_M | 0.25GB |
| GPT2_sft_and_dpo_tuned.Q4_1.gguf | Q4_1 | 0.25GB |
| GPT2_sft_and_dpo_tuned.Q5_0.gguf | Q5_0 | 0.27GB |
| GPT2_sft_and_dpo_tuned.Q5_K_S.gguf | Q5_K_S | 0.27GB |
| GPT2_sft_and_dpo_tuned.Q5_K.gguf | Q5_K | 0.29GB |
| GPT2_sft_and_dpo_tuned.Q5_K_M.gguf | Q5_K_M | 0.29GB |
| GPT2_sft_and_dpo_tuned.Q5_1.gguf | Q5_1 | 0.29GB |
| GPT2_sft_and_dpo_tuned.Q6_K.gguf | Q6_K | 0.32GB |
| GPT2_sft_and_dpo_tuned.Q8_0.gguf | Q8_0 | 0.41GB |
license: apache-2.0 datasets:
This repository GPT-2 Medium model instruct tuned first on the Anthropic-hh dataset and then further aligned on the same dataset with DPO.
import torch
from transformers import GPT2LMHeadModel, GPT2Tokenizer, StoppingCriteria, StoppingCriteriaList
tokenizer_dpo = GPT2Tokenizer.from_pretrained("RaushanTurganbay/GPT2_sft_and_dpo_tuned")
model_dpo = GPT2LMHeadModel.from_pretrained("RaushanTurganbay/GPT2_sft_and_dpo_tuned")
class StoppingCriteriaSub(StoppingCriteria):
def __init__(self, stops=[], encounters=1):
super().__init__()
self.stops = [stop.to("cuda") for stop in stops]
def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor):
for stop in self.stops:
if torch.all((stop == input_ids[0][-len(stop):])).item():
return True
return False
def stopping_criteria(tokenizer, stop_words):
stop_words_ids = [tokenizer(stop_word, return_tensors='pt')['input_ids'].squeeze() for stop_word in stop_words]
stopping_criteria = StoppingCriteriaList([StoppingCriteriaSub(stops=stop_words_ids)])
return stopping_criteria
# Generate responses
stopping = stopping_criteria(tokenizer, ["\n\nHuman:"])
prompt = "\n\nHuman: {your_instruction}\n\nAssistant:"
inputs_dpo = tokenizer_dpo(prompt, return_tensors="pt")
outputs_dpo = model_dpo.generate(**inputs_dpo, stopping_criteria=stopping, max_length=150)
print("Model Response:", tokenizer_dpo.batch_decode(outputs_dpo))