Downloads · 30 days
1
1% of all-time downloads
gandhiraketla277/finance-llama-3.1-8b-dpo-trained
finance-llama-3.1-8b-dpo-trained is a text generation model from gandhiraketla277. Use it when you need the model to write or continue text. It is set up for peft.
This model was trained using genuine Direct Preference Optimization (DPO) on financial question-answering data. This is TRUE DPO training, not supervised fine-tuning.
Downloads · 30 days
1
1% of all-time downloads
All-time downloads
197
Public
Repo size
185 MB
Likes
0
Public
Click a slice to open those files.
.safetensors168 MB · 91%
From the Hugging Face model README
This model was trained using genuine Direct Preference Optimization (DPO) on financial question-answering data. This is TRUE DPO training, not supervised fine-tuning.
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch
# Load base model
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16
)
base_model = AutoModelForCausalLM.from_pretrained(
"gandhiraketla277/finance-llama-3.1-8b-merged",
quantization_config=bnb_config,
device_map="auto",
torch_dtype=torch.bfloat16
)
# Load TRUE DPO trained adapters
model = PeftModel.from_pretrained(base_model, "gandhiraketla277/finance-llama-3.1-8b-dpo-trained")
tokenizer = AutoTokenizer.from_pretrained("gandhiraketla277/finance-llama-3.1-8b-merged")
# Generate response
prompt = "What is the best investment strategy for beginners?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
Thanks to TRUE DPO training, this model should provide:
This model represents TRUE DPO training with preference optimization, not supervised fine-tuning! 🎯