lab21

Lab 21 — Qwen3.5-4B LoRA Adapter for Vietnamese CSKH Triage

Fine-tuned LoRA adapter on unsloth/Qwen3.5-4B for Customer Support (CSKH) ticket classification and 4-field JSON triage extraction.

🎯 Model Details

• Base Model: unsloth/Qwen3.5-4B
• Adapter Type: LoRA (PEFT)
• Target Modules: All text decoder linear layers (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)
• Rank $r$: 16 ($2 imes r = 32$ LoRA Alpha)
• Learning Rate: $1 imes 10^{-4}$ ($10 imes$ Full-FT scale)
• Task: 4-field JSON Triage (intent, urgency, product, sentiment)

📊 Performance

• Target Accuracy: 97.5% (Baseline a: 0.0%, Baseline b: 76.0%)
• Format Compliance: 100% valid JSON

🚀 Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model_id = "unsloth/Qwen3.5-4B"
adapter_id = "mixp123/lab21"

tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)
model = PeftModel.from_pretrained(model, adapter_id)

prompt = "<|im_start|>system\nPhân loại ticket sau.<|im_end|>\n<|im_start|>user\nShop ơi, mình đặt chuột không dây VN232232. Cho tôi trả lại. Gấp.<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))