qwen3.5-4b-astcss-t5

qwen3.5-4b-astcss-t5

English → astcss selector (sitting_duck's ast_select) at 4B, trained on the tier-5 corpus. The strongest model in this project on both evals, including relational and compound selectors.

Scores

Measured on this merged artifact in float16 — the precision it ships in, not the NF4 the adapter was fitted in:

Tier breakdown on the 108-pair eval: 19/21, 22/25, 29/31, 28/31.

The merge is not identity and was checked rather than assumed: verify_merge reports 248 targeted tensors at max |Δ| 0.045 against 178 untouched at 0.00049, the fp16 round-trip floor. Unlike the previous 4B, which lost a pair on each eval in the merge, this one loses nothing.

Scoring is by execution — the selector runs against the fixture and the returned node set is compared to the reference, so a different string selecting the same nodes counts as correct.

What changed against the previous 4B: the corpus, not the model

Same base, same recipe, same size. The training corpus is the tier-5 one (6,053 pairs / 15,928 rows with per-language cards) rather than the older, smaller set. That is worth +25.4 points on eval_t5 and +1.8 on the near-saturated 108-pair eval.

For scale: a 0.8B trained on the same corpus (qwen3.5-0.8b-astcss-t5) scores 83.6 % on eval_t5, while a 4B on the old corpus scores 65.5 %. Five times the parameters lost to the corpus by 18 points. This model is what happens when you stop trading one against the other.

It also scores above Claude Haiku 4.5 prompted cold on the same 55 pairs (87.3 %). That run's metadata does not record which card haiku was given, so read it as "cold haiku on this eval", not as a card-matched comparison.

It needs the card

Every langcard-trained model measured in this project scores 0.0 % without its system prompt, at every size tested. This artifact was not itself measured card-free; it was trained the same way, so assume it needs one. Trained with a per-language vocabulary card in the system position. Cards live in the astcss-eval repo under train/cards/card_<lang>.md.

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

m = "teaguesterling/qwen3.5-4b-astcss-t5"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m, dtype=torch.float16, device_map="cuda")

card = open("train/cards/card_python.md").read()   # REQUIRED
msgs = [{"role": "system", "content": card},
        {"role": "user", "content": "functions that call range without any try block"}]
ids = tok(tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False),
          return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=48, do_sample=False)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))
# .fn:has(.call#range):not(:has(.try))

Greedy decoding, thinking off, ~5 completion tokens per answer on the 108-pair eval and ~10 on eval_t5.

What it is

QLoRA (r=16, α=32, all-linear) on Qwen3.5-4B in NF4, 2 epochs, per-language card, template cap 8, lr 2e-4, seed 18. Merged into the base weights — no adapter at serving time.

Epoch 2 is the checkpoint published here, and it matters: epoch 1 scored 87.0 % / 83.6 %, so the second epoch was worth +3.7 and +5.5 points with val loss 0.0726 → 0.0553.

Limitations

• Text-only. 312 base tensors are not written: the vision tower (model.visual.*) and the multi-token-prediction head (mtp.*) are dropped by save_pretrained under AutoModelForCausalLM.
• Needs the card. Not a general chat model. Training on a corpus deliberately mixed with card-free examples did not remove this dependency — the card was still worth +13.9 points to a model explicitly taught to work without one.
• NF4-fitted, fp16-merged. The scores above are fp16, so they are the artifact's own; NF4 inference may differ slightly in the other direction.
• eval_t5 is 55 pairs. A 1-pair difference is 1.8 points, so treat small gaps as ties.