qwen3-1.7b-forbidden-constraint-holder

Qwen3-1.7B — Forbidden-Constraint Holder

A QLoRA fine-tune of Qwen/Qwen3-1.7B trained to hold negative constraints while completing an ordinary writing task — "write this, and never use the letter e / the word 'however' / any quotation marks" — including when the task itself pulls toward the banned token.

Every constraint has a deterministic checker, so adherence here is measured, not judged: an LLM judge scores task quality only and can never overturn a checker.

Read the results section before using this model. It improves on the constraint families it was trained on and regresses on families it was not. That trade is the finding, and it is stated in full below.

Provenance

This repo holds the merged fp16 model at the root and the LoRA adapter under adapter/.

Training data

Teacher-distilled and rejection-filtered through the same deterministic checkers used at evaluation: a generated completion that violated its own constraint set was discarded rather than trained on. Constraint types held out for generalization testing appear in zero training rows.

Hyperparameters

Loss is computed on the completion only; prompt tokens are masked to -100.

Prompt format

Use the chat template with a single user turn and no system message. Training used no system turn, and every training prompt was re-rendered through the evaluation harness's own renderer and compared byte-for-byte before training began, so the training condition equals the evaluation condition. Adding a system prompt puts the model off-distribution.

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "HankH18/qwen3-1.7b-forbidden-constraint-holder"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto", device_map="auto")

messages = [{"role": "user", "content": "Write a short paragraph about the ocean. Do not use the letter 'e' anywhere in your response."}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True,
                                 enable_thinking=False, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(inputs, max_new_tokens=512, do_sample=False)[0][inputs.shape[-1]:]))

enable_thinking=False matters: the training rows were rendered with it off, and Qwen3's template emits a thinking block otherwise.

Results

Eval set data/scenarios/corpus.jsonl, judge claude-sonnet-4-6, rubric hash 12a0dcaaf29080cb. spec_pass = every checker passes and judge task-quality >= 3. robustness is the same quantity over the adversarial subset only.

Scored rows: base 146, tuned 147. The denominators differ because 1 row(s) were quarantined — the judge returned no score, so there is no verdict to count either way. Rates are over scored rows.

Violation rate per constraint type — lower is better. Types marked held out appear in zero training rows.

Limitations, stated plainly

• It does not reliably hold a banned letter. That is the hardest constraint in the set and the checker requires zero occurrences. The tuned model reduces how often the banned letter appears — the median response goes from 34 occurrences to 14 — but a response with 14 is still a failure, and the pass rate stays near the floor.
• It gets worse at constraint families it never saw. Compare the held-out rows in the table above. Fine-tuning bought compliance on the trained families and spent generalization on the untrained ones. If your constraint type is not in the training list, the base model is the better choice.
• Single user turn, no system prompt. See the prompt format above.
• 1.7B parameters, English only, short-form writing tasks.

Reproducing these numbers

python eval.py --model hf:HankH18/qwen3-1.7b-forbidden-constraint-holder@<commit> \
               --compare-with hf:Qwen/Qwen3-1.7B@70d244cc86ccca08cf5af4e1e306ecf908b1ad5e \
               --eval-set data/scenarios/corpus.jsonl

The harness, the checkers, the scenario corpus and the raw judge transcripts are all in the project repo.