qwen3-4b-lora-y-v40

qwen3-4b-lora-y-v40

LoRA adapter fine-tuned for structured output tasks (JSON / YAML / XML / TOML / CSV).

This repository contains adapter weights only. Load with the base model separately.

Training Summary

• Base model: Qwen/Qwen3-4B-Instruct-2507
• Dataset file: merged_dataset_final_clean_v41.jsonl
• Max sequence length: 512
• Epochs: 2
• Learning rate: 3.896385201823815e-06
• Warmup ratio: 0.06105633616739185
• Weight decay: 0.0
• LoRA rank/alpha: 128 / 128
• LoRA dropout: 0.0
• Gradient accumulation: 4
• LoRA target modules: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
• Seed: 73259
• System policy mode: off

Local Reference Evaluation

Local evaluation on public_150.json (for offline comparison only):

• Parse rate: 141/150 (94.0%)
• Proxy score: 0.6440
• Key coverage avg: 55.2%
• Format parse rates: JSON 100%, YAML 100%, TOML 68%, XML 95%, CSV 100%

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model_id = "Qwen/Qwen3-4B-Instruct-2507"
adapter_model_id = "yamaTK/qwen3-4b-lora-y-v40"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_model_id)

Notes

• Training data follows competition constraints (provided datasets only, no LLM-generated data).
• public_150.json was not used for training data generation.