Downloads · 30 days
5
19% of all-time downloads
TuanPham2308/lab21-qwen3.5-2b-cskh-triage-lora
lab21-qwen3.5-2b-cskh-triage-lora is a text generation model from TuanPham2308. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
Educational LoRA adapter from a fine-tuning lab (AICB course, Day 21). Trained to convert Vietnamese customer-support tickets into a 4-field JSON triage (intent, urgency, product, sentiment).
Downloads · 30 days
5
19% of all-time downloads
All-time downloads
27
Public
Repo size
87.3 MB
Likes
0
Public
Click a slice to open those files.
.safetensors67.3 MB · 77%
From the Hugging Face model README
Educational LoRA adapter from a fine-tuning lab (AICB course, Day 21). Trained to
convert Vietnamese customer-support tickets into a 4-field JSON triage
(intent, urgency, product, sentiment).
Full report, all raw results, and the training/eval pipeline: https://github.com/phamquoctuan2308/Day21-Track3-2a202601983-PhamQuocTuan
This adapter was trained for 2 epochs on 225 narrow-domain examples with no replay data. It scores excellently on the target task but has catastrophic forgetting of general capability:
| Metric | Base + optimized prompt | This adapter |
|---|---|---|
| Target task accuracy | 0.595 | 0.995 |
| General-knowledge regression score | 0.689 | 0.067 |
| JSON format validity | 1.000 | 1.000 |
11/15 general-knowledge questions in the regression eval get a JSON triage object
back instead of an answer — e.g. asking "Ai là tác giả của Truyện Kiều?" ("Who wrote
Truyện Kiều?") returns {"intent": "hoi", "urgency": "trung_binh", ...} instead of
"Nguyễn Du". The model has effectively lost the ability to do anything except emit
this JSON schema. See the linked report §5–6 for the full regression-gate analysis
(verdict: FAILED) and the fix that was not applied here (1–5% general-purpose
replay data mixed into training, per LoRA-fine-tuning best practice).
Qwen/Qwen3.5-2Br=16, alpha=32, target modules = all linear layers (text-linear
placement), 16,819,200 trainable params1e-4 (cosine schedule), 2 epochs, 58 optimizer stepsper_device_batch=1 × grad_accum=8)assistant-only (loss computed only on the JSON answer span,
supervised_fraction=0.394 — verified by decoding the supervised span, not
assumed)from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-2B", dtype="bfloat16")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-2B")
model = PeftModel.from_pretrained(base, "TuanPham2308/lab21-qwen3.5-2b-cskh-triage-lora")
Framework: PEFT 0.20.0