jev-typed-decisions-causal-0.6b

Cached causal typed scorer (arm B, Jev-like reproduction)

LoRA adapters for Qwen/Qwen3-0.6B-Base trained with the arm-B recipe from pngwn/typed-decisions-causal-experiment (a cached causal typed scorer, in the style of the recently released "Jev" model).

Recipe (exactly the report's arm B)

• Base: Qwen/Qwen3-0.6B-Base (596.0M params), LM head frozen — only LoRA adapters train
• LoRA: r=16, alpha=32, dropout=0.05, targets q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
• Encoding: branch mode — prompt + suffix_i + letter_i, loss read at the single answer-letter position, restricted CE over the candidate option-letter tokens of the frozen LM head (never materialising full-vocab logits)
• Corpus: pngwn/typed-decisions-v2 @ 74a8ed2d0d28955351805f7829d7149961172f88 (train 31,109 states / 45,932 decisions)
• Optimizer: AdamW, lr 1e-4, wd 0, 3% linear warmup + cosine, clip 1.0, batch 16, max_len 1024, seed 20260916
• Steps: 5,742 (2 full epochs) — same config as the report's arm-B run
• Wall clock: 37 min 21 s on a100-large (torch 2.14.0+cu130, transformers 5.17.0, peft 0.21.0)
• Training metrics: Trackio dashboard

Observed results

Final calibration-split eval (n=640 decisions, quick_eval health metric): acc 0.6234, NLL 1.2755 (eval acc climbed 0.473 @ step 200 → 0.623 @ end; train loss 1.33 → ~0.4).

Note: this is the in-training quick-eval on a cal subsample, not the report's matched 5,214-decision test harness with temperature scaling — the report's full-eval numbers for arm B (accuracy 0.7518 on test, ECE 0.0154 after temperature scaling) were produced by a separate evaluation pipeline not re-run here.

Usage

Load Qwen/Qwen3-0.6B-Base + these adapters, prefill a typed-decision prompt once, then read the next-token distribution restricted to the candidate option letters (" A".." Z" are single tokens in the Qwen tokenizer) — one shared KV cache, one branch per decision.