Downloads · 30 days
8
1% of all-time downloads
iamhyunmin/gemma2-9b-deita-sft-epoch3-dpo
gemma2-9b-deita-sft-epoch3-dpo is a text generation model from iamhyunmin. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as gemma.
This repository contains the epoch-2 inference export of a DPO fine-tune of iamhyunmin/gemma2-9b-deita-sft-epoch3. The base model is derived from Google Gemma 2 9B.
Downloads · 30 days
8
1% of all-time downloads
All-time downloads
697
Public
Parameters
4.6B
40.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors40.6 GB · 100%
From the Hugging Face model README
This repository contains the epoch-2 inference export of a DPO fine-tune of
iamhyunmin/gemma2-9b-deita-sft-epoch3.
The base model is derived from Google Gemma 2 9B.
The model weight files in this repository were modified from the base model by full-parameter Direct Preference Optimization (DPO). This is a model derivative and is not an official Google model. Google has not endorsed this model.
argilla/dpo-mix-7k (MIT)5e-70.1The training prompt used John as the user role tag and the system prompt
You are a helpful assistant for John.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "iamhyunmin/gemma2-9b-deita-sft-epoch3-dpo"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype="auto",
device_map="auto",
)
messages = [
{"role": "user", "content": "Explain direct preference optimization."}
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
This model is a Gemma Model Derivative and is distributed under and subject to
the Gemma Terms of Use, including the
Gemma Prohibited Use Policy.
By downloading, using, modifying, or distributing this model, you agree to
those terms and restrictions. See NOTICE.
This model may produce inaccurate, biased, unsafe, or otherwise undesirable content. It has not been evaluated for every downstream application. Users are responsible for evaluating the model and complying with applicable laws and the Gemma Terms of Use.