Downloads · 30 days
7
8% of all-time downloads
ryo-llm/dpo-qwen-cot-merged-202602081200
dpo-qwen-cot-merged-202602081200 is a text generation model from ryo-llm. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This model is a fine-tuned version of Qwen/Qwen3-4B-Instruct-2507 using Direct Preference Optimization (DPO) via the Unsloth library.
Downloads · 30 days
7
8% of all-time downloads
All-time downloads
84
Public
Parameters
4.1B
4.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors4.1 GB · 100%
How the weights are stored.
U83.1B · 76%
From the Hugging Face model README
This model is a fine-tuned version of Qwen/Qwen3-4B-Instruct-2507 using Direct Preference Optimization (DPO) via the Unsloth library.
This repository contains the full model weights (ready for inference with transformers).
This model has been optimized using DPO to align its responses with preferred outputs, focusing on structured response quality and reducing unwanted preface text.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "ryo-llm/dpo-qwen-cot-merged-202602081200"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.float16,
device_map="auto",
trust_remote_code=True,
)
prompt = "Your question here"
inputs = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}],
tokenize=True,
add_generation_prompt=True,
return_tensors="pt",
).to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))