Downloads · 30 days
8
4% of all-time downloads
koguma-ai/sft-dpo-qwen-cot-merged0207
sft-dpo-qwen-cot-merged0207 is a text generation model from koguma-ai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This model is a fine-tuned version of Qwen/Qwen3-4B-Instruct-2507 using a two-stage training pipeline: SFT (Supervised Fine-Tuning) followed by DPO (Direct Preference Optimization) via the Unsloth library.
Downloads · 30 days
8
4% of all-time downloads
All-time downloads
190
Public
Parameters
4B
8.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors8 GB · 100%
From the Hugging Face model README
This model is a fine-tuned version of Qwen/Qwen3-4B-Instruct-2507 using a two-stage training pipeline: SFT (Supervised Fine-Tuning) followed by DPO (Direct Preference Optimization) via the Unsloth library.
This repository contains the full-merged 16-bit weights. No adapter loading is required.
The base model was first fine-tuned with SFT to learn structured output generation and Chain-of-Thought reasoning.
The SFT-trained LoRA was merged into the base model, then a new LoRA adapter was applied for DPO training to further align the model with preferred outputs.
Since this is a merged model, you can use it directly with transformers.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "koguma-ai/sft-dpo-qwen-cot-merged0207"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.float16,
device_map="auto"
)
prompt = "Your question here"
inputs = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}],
tokenize=True,
add_generation_prompt=True,
return_tensors="pt"
).to("cuda")
outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0]))