Downloads · 30 days
5
9% of all-time downloads
tussiiiii/qwen3-4b-structured-output-lora-continued-v5-daichira-ver2-dpo-ver3
qwen3-4b-structured-output-lora-continued-v5-daichira-ver2-dpo-ver3 is a text generation model from tussiiiii. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This repository provides a fully merged model created by sequentially merging: Base Model → SFT LoRA → DPO LoRA.
Downloads · 30 days
5
9% of all-time downloads
All-time downloads
53
Public
Parameters
4B
8.6 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors8.6 GB · 100%
From the Hugging Face model README
This repository provides a fully merged model created by sequentially merging: Base Model → SFT LoRA → DPO LoRA.
This repository contains:
To use this model: Load it directly with transformers (no PEFT adapters).
This adapter was optimized using DPO to align responses to preferred outputs, focusing on improving reasoning style (Chain-of-Thought preferences) and structured response quality based on the provided preference dataset.
DPOTrainer (Unsloth patched)Load the merged model directly (no PEFT adapters needed).
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "your_id/your-repo-name" # this merged repo
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.float16,
device_map="auto",
trust_remote_code=True,
)
# Test inference
prompt = "Your question here"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))