Downloads · 30 days
4
9% of all-time downloads
zephyr-7b-dpo_dataset-mix
jmajkutewicz/zephyr-7b-dpo_dataset-mix
zephyr-7b-dpo_dataset-mix is a text generation model from jmajkutewicz. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
This repo contains LoRA adapter created by aligning Zephyr 7B SFT using Direct Preference Optimization (DPO) on the mix all following datasets: OpenAssistant Conversations Dataset (OASST1) Anthropic HH-RLHF
All-time downloads
47
Public
Repo size
1.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors336 MB · 99%
At a glance
- Task
- Text Generation
- Library
- peft
- License
- apache-2.0
- Model type
- mistral
- Access
- Public
- Created
- Sep 26, 2025
- Updated
- Sep 26, 2025
- SHA
- 9ae5e0a8
Try a prompt
Base models
Model card
From the Hugging Face model README
Zephyr 7B SFT aligned with DPO on mix of datasets with β=0.01
This repo contains LoRA adapter created by aligning Zephyr 7B SFT using Direct Preference Optimization (DPO) on the mix all following datasets:
It was trained as a series of models for studying DPO alignment.
Model details
- Base model: alignment-handbook/zephyr-7b-sft-full
- Preference datasets:
- DPO beta: 0.01
- Training framework: PEFT/LoRA
See the base model card for usage and chat template details.
Training hyperparameters
- Epochs: 1
- Batch size: 16
- Learning rate: 5e-06
- Learning rate scheduler: cosine
- Learning rate warmup ratio: 0.1
- Gradient accumulation: 2
- LoRA:
- rank: 64
- alpha: 64
- dropout: 0.05
- target modules: [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj]
License
This adapter is released under the Apache License 2.0.
Citation
If this work was helpful, please cite:
TBA
- Task
- Text Generation
- Library
- peft
- Type
- mistral
- License
- apache-2.0
- Languages
- en
- Created
- Sep 26, 2025
- Updated
- Sep 26, 2025