qwen2.5-1.5b-weather-intent-v2

qwen2.5-1.5b-weather-intent-v2

QLoRA adapter for Qwen/Qwen2.5-1.5B-Instruct — a production-hardened weather-intent parser (v2). Maps a user message to a structured JSON intent, and rejects what it can't answer instead of confabulating.

v2 over the v1 SFT baseline (which hit 98.6% exact-match but confabulated on out-of-scope input) adds three independently-measured levers:

• reject training — an explicit intent: "out_of_scope" class + boundary-negative data (fixes confabulation)
• grammar-constrained decoding — XGrammar JSON-schema, making out-of-vocabulary values impossible (fixes surface form)
• a deterministic gate — route() decides accept vs escalate from the structured output (model-cascade)

{"intent": "weather", "location": "Paris", "date": "this_weekend", "metric": "rain", "time_of_day": null}
{"intent": "out_of_scope", "location": null, "date": null, "metric": null, "time_of_day": null}

Schema

Results (300 held-out; base few-shot, ft/ft+grammar zero-shot; greedy)

The honest read: reject-training does the heavy lifting (ft alone drops false-accept to 0); grammar is the last-mile guarantee that drives out-of-vocab to a structural 0. The gate turns structure into a cheap-over-escalation routing decision. Grammar guarantees validity, not correctness — the constraint tax is measured, not assumed.

Training

• QLoRA (4-bit NF4, double-quant, bf16), LoRA r=16 / alpha=32 on all linear layers
• 1 epoch, effective batch 16, LR 0.0002 cosine, assistant-only loss
• 1500 examples (25% out-of-scope, 4 balanced strata); single RTX 3050 Ti (4 GB)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen2.5-1.5B-Instruct"
tok = AutoTokenizer.from_pretrained(base)
model = PeftModel.from_pretrained(AutoModelForCausalLM.from_pretrained(base, device_map="auto"), "Nicholas55555/qwen2.5-1.5b-weather-intent-v2")

sys = "You extract structured intent from user messages about weather. Return ONLY a JSON object..."
msgs = [{"role": "system", "content": sys}, {"role": "user", "content": "how's the pollen in Austin?"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
print(tok.decode(model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=64)[0], skip_special_tokens=True))
# -> {"intent": "out_of_scope", ...}   (pollen is not a supported metric)

For guaranteed-valid output, pair with XGrammar constrained decoding against the schema.