Skip to content

jmajkutewicz

zephyr-7b-dpo_PKU-SafeRLHF

jmajkutewicz/zephyr-7b-dpo_PKU-SafeRLHF

zephyr-7b-dpo_PKU-SafeRLHF is a text generation model from jmajkutewicz. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.

This repo contains LoRA adapter created by aligning Zephyr 7B SFT on the PKU-SafeRLHF dataset using Direct Preference Optimization (DPO). It was trained as a series of models for studying DPO alignment.

Downloads · 30 days

6

18% of all-time downloads

All-time downloads

33

Public

Repo size

1.3 GB

Likes

0

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors336 MB · 99%

At a glance

Task
Text Generation
Library
peft
License
apache-2.0
Model type
mistral
Access
Public
Created
Sep 26, 2025
Updated
Sep 26, 2025
SHA
f2ffc840

Try a prompt

Base models

Task
Text Generation
Library
peft
Type
mistral
License
apache-2.0
Languages
en
Created
Sep 26, 2025
Updated
Sep 26, 2025