Downloads · 30 days
0
qwen3-8b-dpo-lmsys
subbuc/qwen3-8b-dpo-lmsys
qwen3-8b-dpo-lmsys is a machine learning model from subbuc. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This model is a DPO fine-tuned version of Qwen3-8B-4bit, trained on the LMSYS Arena Human Preference dataset.
Access
Public
Updated Nov 12, 2025
Repo size
196 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt123 MB · 44%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Nov 12, 2025
- Updated
- Nov 12, 2025
- SHA
- f96df14f
Base models
Model card
From the Hugging Face model README
Qwen3-8B DPO LMSYS
This model is a DPO fine-tuned version of Qwen3-8B-4bit, trained on the LMSYS Arena Human Preference dataset.
Training Details
- Base Model: unsloth/Qwen3-8B-4bit
- Training Method: Direct Preference Optimization (DPO)
- Dataset: LMSYS Arena Human Preference 55k
- Training Steps: 60
- Beta: 0.1
MT-Bench-101 Results
- DPO Score: Check results.json
- Improvement over SFT baseline: Check results.json
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("subbuc/qwen3-8b-dpo-lmsys")
tokenizer = AutoTokenizer.from_pretrained("subbuc/qwen3-8b-dpo-lmsys")
# Your inference code here
- License
- apache-2.0
- Languages
- en
- Created
- Nov 12, 2025
- Updated
- Nov 12, 2025