Qwen3-4B-Instruct-2507-PII-SFT-LoRA

Qwen3-4B-Instruct-2507 — PII Masking (SFT LoRA)

LoRA adapter that fine-tunes Qwen/Qwen3-4B-Instruct-2507 to detect personally identifiable information (PII) in text, replace it with [PII] tags, and return the masked text wrapped in <masked_output>...</masked_output>.

Results

Greedy exact-match on a 5,914-row held-out split (provably disjoint from training by input):

*mean_reward is the normalized reward (exact·1.0 + pii_count·0.5 + format·0.1) / 1.6 (max 1.0). The Prime Intellect verifier for this task reports the same three components un-normalized (max 1.6), so e.g. a Prime reward of ~1.50 corresponds to ~0.94 here.

Format compliance is already ~1.00 on the base model; the adapter's gain is almost entirely learning the exact [PII] segmentation convention (it cuts the relaxed-minus-exact gap from 0.168 to 0.045).

Training

• Method: supervised fine-tuning, completion-only loss (loss on the gold masked answer + EOS only).
• LoRA: r=8, α=16, dropout 0, all-linear (q/k/v/o/gate/up/down projections).
• Data: 5,000 examples from AdamLucek/open-pii-masking-en-us-30k (train split).
• Schedule: 2 epochs, lr 2e-4 cosine, bf16, gradient checkpointing.

The published adapter was extracted from the merged SFT checkpoint by a per-layer rank-8 SVD of (W_merged − W_base). Base + this adapter reproduces the merged model's held-out exact-match to within bf16 rounding (0.9319 vs 0.9325).

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-4B-Instruct-2507", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "ichetandhembre/Qwen3-4B-Instruct-2507-PII-SFT-LoRA")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")

SYSTEM = ("Replace all personally identifiable information (PII) in the text with [PII] tags. "
          "PII includes: names, dates, phone numbers, SSNs, account numbers, addresses, "
          "email addresses. Wrap the masked text in <masked_output>...</masked_output>.")
msgs = [{"role": "system", "content": SYSTEM},
        {"role": "user", "content": "Hi, this is John Smith, call me at 555-0123."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=384, do_sample=False)[0][ids.shape[1]:],
                 skip_special_tokens=True))

Or serve base + adapter with vLLM (enable_lora=True, max_lora_rank>=8).