qwen3-4b-struct-output

Qwen3-4B Structured Output (SFT + DPO)

This model is a full-merged version of Qwen/Qwen3-4B-Instruct-2507 fine-tuned for structured output tasks (JSON / YAML / XML / TOML / CSV) via a two-stage pipeline: SFT (QLoRA) → DPO.

Training Pipeline

Qwen/Qwen3-4B-Instruct-2507  (base)
  |
  v
SFT via QLoRA (4-bit, Unsloth)
  - Dataset: v2 + structured-hard-sft-4k + structured-5k-mix-sft (concatenated)
  - Adapter saved as LoRA weights
  |
  v
Merge LoRA adapter into full 16-bit weights
  |
  v
DPO (Direct Preference Optimization)
  - Dataset: u-10bei/dpo-dataset-qwen-cot
  |
  v
GawinGowin/qwen3-4b-struct-output  (this model)

SFT Configuration

DPO Configuration

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "GawinGowin/qwen3-4b-struct-output"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.float16, device_map="auto",
)

Sources & License

• Base model: Qwen/Qwen3-4B-Instruct-2507
• SFT data: u-10bei/structured_data_with_cot_dataset_512_v2, daichira/structured-hard-sft-4k, daichira/structured-5k-mix-sft — MIT License
• DPO data: u-10bei/dpo-dataset-qwen-cot — MIT License
• License: Apache 2.0
• Compliance: Users must comply with the MIT license of the training datasets and the original terms of use of the base model.