Downloads · 30 days
12
22% of all-time downloads
elguantletai002/checklist-reviewer-qwen3-0.6b-n455
checklist-reviewer-qwen3-0.6b-n455 is a machine learning model from elguantletai002. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
This is the checkpoint every reported number refers to. Trained on data/sftv2.jsonl, evaluated on eval/sets/reviewv2.jsonl, whose fixtures share no distinctive line with the training seeds (test-enforced).
Downloads · 30 days
12
22% of all-time downloads
All-time downloads
55
Public
Repo size
657 MB
Likes
0
Public
Click a slice to open those files.
.safetensors646 MB · 98%
From the Hugging Face model README
This is the checkpoint every reported number refers to. Trained on
data/sft_v2.jsonl, evaluated on eval/sets/review_v2.jsonl, whose fixtures
share no distinctive line with the training seeds (test-enforced).
Diff in, checklist JSON out. Ten frozen rules, no praise, no patches, no naming the symbol to change.
Judged by claude-sonnet-5, frozen rubric, 36 scenarios.
| base | this adapter | prompted frontier ceiling | |
|---|---|---|---|
| spec-adherence | 0.139 | 0.611 | 0.861 |
| robustness | 0.208 | 0.583 | 0.875 |
| strict (right rule, legal anchor) | 0.028 | 0.278 | 0.806 |
| false-clean (lower is better) | 0.133 | 0.100 | 0.000 |
Data closed roughly half the gap on adherence and a third on strict. It reaches neither the 0.90 bar nor the prompted ceiling.
What it learned: the format completely — markdown fences went from 32 of 36 replies to zero — and the rule vocabulary.
What it did not: where to point. strict reached 0.278 because the model
did not learn to count @@ hunk arithmetic. That is the dominant residual
failure and it is also the only genuine failure mode the frontier shows.
It did not learn to say nothing. false-clean fell (0.133 → 0.100) while
adherence quadrupled. {"findings": []} passes every schema and groundedness
check, so a model that collapsed to empty would score well and be worthless.
LoRA — not QLoRA: bf16, no quantisation, since 4-bit saves VRAM that is not
scarce for a 0.6B model on a 24GB A10. r=256, alpha=256,
target_modules="all-linear", lr 2e-4, 2 epochs, effective batch 16, on 455
teacher-distilled and filtered examples.
Known defect: TRL 0.15 exposes neither assistant_only_loss nor
completion_only_loss, so loss covered the ~1.1k-token system prompt too. Most
of the gradient went to reproducing the spec, which is likely part of why
strict lagged.
Sibling checkpoints n227, n113, n56 form the data-efficiency curve;
minimum viable N is 227.
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B")
model = PeftModel.from_pretrained(base, "elguantletai002/checklist-reviewer-qwen3-0.6b-n455").merge_and_unload()
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")
The system prompt is spec/behavior_spec.md from the source repo. Behaviour is
undefined without it.
The
n414/n207/n103/n51repos are superseded: they were trained on a dataset that shared fixtures with its own eval set, so their numbers measured recognition of training data. Use this one.