qwen3.5-9b-astcss

qwen3.5-9b-astcss

English → astcss selector (sitting_duck's ast_select) at 9B, merged into the full Qwen3_5ForConditionalGeneration checkpoint so it keeps the base's own layout — vision tower, nested config, preprocessor files and all.

Scores

Measured on this merged artifact, in NF4:

Tier breakdown on the 108-pair eval: 19/21, 22/25, 26/31, 28/31.

The merge reproduces the adapter exactly on both evals. Scoring is by execution — the selector runs against the fixture and the returned node set is compared to the reference, so a different string selecting the same nodes counts as correct.

Read this before reaching for the biggest model

A 4B trained on the same corpus with the same recipe (qwen3.5-4b-astcss-t5) scores 90.7 % / 90.9 % — better than this on both evals, at less than half the size.

That was measured properly: an identical 9B run on the same corpus (same epochs, accum, lr, seed) reached 87.0 % / 87.3 %, so the gap is the model and not the data. The 9B also had the better validation loss at both epochs (0.0453 vs 0.0553) while losing on both evals — worth knowing if you are selecting checkpoints by loss.

This model exists because some targets can only run a 9B. If yours can run a 4B, use the 4B.

It needs the card

Every card-trained model measured in this project scores 0.0 % without its system prompt, at every size tested. Trained with a per-language vocabulary card in the system position. Cards live in the astcss-eval repo under train/cards/card_<lang>.md.

from transformers import AutoTokenizer, AutoModelForImageTextToText
import torch

m = "teaguesterling/qwen3.5-9b-astcss"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForImageTextToText.from_pretrained(m, dtype=torch.bfloat16, device_map="cuda")

card = open("train/cards/card_python.md").read()   # REQUIRED
msgs = [{"role": "system", "content": card},
        {"role": "user", "content": "functions that call range without any try block"}]
ids = tok(tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False),
          return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=48, do_sample=False)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))
# .fn:has(.call#range):not(:has(.try))

Greedy decoding, thinking off, ~5 completion tokens per answer.

What it is

QLoRA (r=16, α=32, all-linear — attention, MLP and the linear-attention projections) on Qwen3.5-9B in NF4, 1 epoch, per-language card. Folded into a copy of the base checkpoint rather than re-serialised through a CausalLM head: 248 of 775 tensors changed, 527 byte-identical to the base, architectures: ["Qwen3_5ForConditionalGeneration"], bf16, 19.3 GB.

Limitations

• Needs the card. Not a general chat model.
• Beaten by a 4B on both evals — see above.
• NF4-fitted, bf16-merged. The scores above are NF4, the same stack the adapter was measured in; bf16 inference may differ slightly.
• Selector-only. It emits ast_select selectors. It does not edit code.
• eval_t5 is 55 pairs, so a 1-pair difference is 1.8 points — treat small gaps as ties.