Downloads · 30 days
3
11% of all-time downloads
jmajkutewicz/Llama-3.1-Tulu-3-8B-DPO_dataset-mix
Llama-3.1-Tulu-3-8B-DPO_dataset-mix is a text generation model from jmajkutewicz. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as llama3.1.
This repo contains LoRA adapter created by aligning Tülu3 8B using Direct Preference Optimization (DPO) on the mix all following datasets: OpenAssistant Conversations Dataset (OASST1) Anthropic HH-RLHF
Downloads · 30 days
3
11% of all-time downloads
All-time downloads
27
Public
Repo size
1.3 GB
Likes
0
Public
Click a slice to open those files.
.safetensors336 MB · 97%
From the Hugging Face model README
This repo contains LoRA adapter created by aligning Tülu3 8B using Direct Preference Optimization (DPO) on the mix all following datasets:
It was trained as a series of models for studying DPO alignment.
See the base model card for usage and chat template details.
This adapter is released under Meta's Llama 3.1 Community License Agreement. Llama 3.1 is © Meta Platforms, Inc.
If this work was helpful, please cite:
TBA