Downloads · 30 days
6
14% of all-time downloads
NobutaMN/qwen3-4b-structeval-lora-v2change-sft7000-550DPO-run7henkou-2shusei
qwen3-4b-structeval-lora-v2change-sft7000-550DPO-run7henkou-2shusei is a text generation model from NobutaMN. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
This model is a fine-tuned version of NobutaMN/qwen3-4b-structeval-merged-v2change-sft7000-run7 using Direct Preference Optimization (DPO) via the Unsloth library.
Downloads · 30 days
6
14% of all-time downloads
All-time downloads
44
Public
Repo size
77.6 MB
Likes
0
Public
Click a slice to open those files.
.safetensors66.1 MB · 81%
From the Hugging Face model README
This model is a fine-tuned version of NobutaMN/qwen3-4b-structeval-merged-v2change-sft7000-run7 using Direct Preference Optimization (DPO) via the Unsloth library.
This repository contains LoRA adapter weights only. The base model must be loaded separately.
This model has been optimized using DPO to align its responses with preferred outputs, focusing on improving reasoning (Chain-of-Thought) and structured response quality based on the provided preference dataset.
This is a LoRA adapter. Load the base model, then apply the adapter.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
from peft import PeftModel
base_id = "NobutaMN/qwen3-4b-structeval-merged-v2change-sft7000-run7"
adapter_id = "your_id/your-repo-name"
tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(
base_id,
torch_dtype=torch.float16,
device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter_id)
# Test inference
prompt = "Your question here"
inputs = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}],
tokenize=True,
add_generation_prompt=True,
return_tensors="pt"
).to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0]))