Downloads · 30 days
5
18% of all-time downloads
jmajkutewicz/Llama-3.1-Tulu-3-8B-DPO_ultrafeedback
Llama-3.1-Tulu-3-8B-DPO_ultrafeedback is a text generation model from jmajkutewicz. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as llama3.1.
This repo contains LoRA adapter created by aligning Tülu3 8B on the UltraFeedback Binarized dataset using Direct Preference Optimization (DPO). It was trained as a series of models for studying DPO alignment.
Downloads · 30 days
5
18% of all-time downloads
All-time downloads
28
Public
Repo size
1.3 GB
Likes
0
Public
Click a slice to open those files.
.safetensors336 MB · 97%
From the Hugging Face model README
This repo contains LoRA adapter created by aligning Tülu3 8B on the UltraFeedback Binarized dataset using Direct Preference Optimization (DPO). It was trained as a series of models for studying DPO alignment.
See the base model card for usage and chat template details.
This adapter is released under Meta's Llama 3.1 Community License Agreement. Llama 3.1 is © Meta Platforms, Inc.
If this work was helpful, please cite:
TBA