Qwen3.5-9B-Humanize-DPO-Round2

Qwen3.5-9B Humanize DPO Round 2

LoRA adapter fine-tuned with DPO on Qwen3.5-9B for Chinese text humanization. This is the latest and most capable version, especially strong on academic and technical Chinese text.

Uses a pure self-play approach: the rejected samples are outputs from an intermediate checkpoint of the previous training stage, teaching the model to consistently surpass its own prior output.

Model Details

What It Does

• Academic papers: preserves all numbers, model names, formulas, and technical terms — verified on 10 academic scenarios (3 samples each)
• Technical reports: maintains technical register without sounding robotic
• Daily text: more concise and natural than Round 1

Usage

from unsloth import FastLanguageModel
from peft import PeftModel

base_model, proc = FastLanguageModel.from_pretrained(
    "unsloth/Qwen3.5-9B", max_seq_length=2048, load_in_4bit=False,
)
tokenizer = proc.tokenizer if hasattr(proc, "tokenizer") else proc

model = PeftModel.from_pretrained(
    base_model, "XiangJinYu/Qwen3.5-9B-Humanize-DPO-Round2", is_trainable=False,
)
if hasattr(model, "config") and getattr(model.config, "model_type", "") == "qwen3_5":
    model.config.model_type = "qwen3"
FastLanguageModel.for_inference(model)

instruction = "请将下面文本改写得更像自然人写作，保持原意与事实，不要加标题或说明。"
text = "本文提出了一种基于U-Net改进的医学影像分割方法，Dice系数达到0.923，较基线方法提升了4.7个百分点，推理速度提升约30%。"
messages = [{"role": "user", "content": [{"type": "text", "text": f"{instruction}\n\n原文：{text}"}]}]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
# Recommended: temperature 0.60-0.65 for academic texts
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.65,
                         top_p=0.9, do_sample=True, repetition_penalty=1.1)
gen = outputs[0][inputs["input_ids"].shape[1]:]
print(tokenizer.decode(gen, skip_special_tokens=True))

Training Details

• DPO data: 2000 pairs — chosen = CSL human academic text, rejected = prior checkpoint outputs (pure self-play, no casual mixing)
• beta: 0.2, lr=5e-7 (conservative to avoid drift), 2 epochs (250 steps)
• Rejected reward: stayed negative and decreasing throughout all 250 steps — no instability
• Key design: pure self-play rejected gives clean, single-distribution gradient signal

Academic Test Results

Tested on 10 academic scenarios (3 samples each), all key numbers preserved:

Note: Use temperature 0.60-0.65 for academic texts. Higher temperatures occasionally cause rare technical term substitutions.

Model Series