rev-qwen3.5-4b

Rev decision model: Qwen3.5-4B (jb_20260923-002700_4b)

A Rev decision model: a frozen Qwen/Qwen3.5-4B backbone with a rank-16 LoRA on every layer's attention and DeltaNet projections (q/k/v/o_proj, in_proj_qkv/z/a/b, out_proj; alpha 32) and a 256-d pointer head. Given a state (a document, JSON record or text), a question and a set of named options, the head scores each option's last token against the decision token and takes a softmax over the offered options only. No tokens are generated: one forward pass returns a choice and a probability per option, for any number of options.

This is the checkpoint of training run jb_20260923-002700_4b.

Calibration temperature 1.96 (stored in the checkpoint metadata; scores are divided by it before the softmax).

Training data

Trained on 19,792 public decisions plus 6,285 synthetic rule-labelled decisions (26,077 rows after the 4,096-token filter and holdout exclusion, 25,886 used): multiple-choice reading and reasoning decisions from public datasets (SocialIQA, CosmosQA, RACE, MultiRC, ContractNLI and others), each rendered as state + question + options, filtered to 4,096 tokens, and each seen in two option orders with a KL consistency term so the answer does not depend on where the right option sits.

The synthetic set (data/synth_jevbench.jsonl.gz, generated by synth_jevbench.py in the repo) covers long policies with lookups, date and quantity arithmetic, multi-hop records, routing, traps, yes/no and ordered-level questions; every label is derived by rule from the generated facts, and a wrong "helpful note" is planted on purpose so the model learns not to copy it.

Not trained on: the frozen evaluation sets in the repo (eval_sets/, including the 975-question public holdout, excluded by question id and by source document), any JevBench item (public or held back), and Jev's official 102-question workflow set. The holdout number above is measured on data the model never saw.

Files

• checkpoint.pt: torch.save dict with adapters ({module_path: {'a': (16, in), 'b': (out, 16)}}, FP32), headq / headk state dicts (weight: (256, hidden)), metadata (this card's numbers) and examples. Merge rule: W += (alpha / rank) * B @ A = 2 * B @ A.
• metadata.json, manifest.json (data checksums and pinned base revisions), status.json, trained_development.json (the holdout run: per-question choices and probabilities).

Serving

The base model is downloaded from the Hub at the pinned revision, the LoRA is merged into its weights, and the heads are loaded. Everything needed is in serve_local.py:

pip install torch transformers accelerate huggingface_hub fastapi uvicorn
# optional, faster DeltaNet layers on CUDA: pip install flash-linear-attention causal-conv1d
git clone https://github.com/robbalian/rev && cd rev
python serve_local.py --repo robbalian/rev-qwen3.5-4b --port 8000

--selftest loads the model, answers one hard-coded request and exits. The Modal deployment with cross-request batching is server.py in the same repo.

Request / response

POST /score takes one state and any number of questions about it; every question is scored in one padded forward.

curl -s http://127.0.0.1:8000/score -H 'content-type: application/json' -d '{
  "state": {"invoice": {"number": "INV-2041", "vendor": "Northwind Traders", "total_usd": 1240.0, "po_match": true}},
  "questions": [
    {"id": "q1", "instructions": "Is the invoice total greater than $500?",
     "criteria": {"yes": "The total exceeds $500", "no": "The total is $500 or less"}},
    {"id": "q2", "instructions": "What should happen to this invoice next?",
     "criteria": {"approve": "Approve for payment", "reject": "Reject and return to vendor", "escalate": "Escalate to a manager"}}
  ]
}'

{"answers": {"q1": {"choice": "yes", "probabilities": {"yes": 0.99, "no": 0.01}},
             "q2": {"choice": "approve", "probabilities": {"approve": 0.9, "reject": 0.02, "escalate": 0.08}}},
 "server_seconds": 0.05, "path": "rows", "rows": [...], "generated_tokens": 0}

state may be any JSON value (serialised compactly) or, since this checkpoint was trained with state_format = raw, a plain string passed through as is. criteria maps option key to description; the keys come back as choice and as the probabilities keys. POST /ping returns the server metadata and GET /health returns {"ok": true}.

The prompt the model sees is

State:
<state>
Question: <instructions>
Options:
<key>: <description>
...
Decision:

and the score for an option is (W_q h_decision) . (W_k h_option) / 16 / T, softmaxed over the offered options.

Source

Code, data, evaluation sets and the full write-up: https://github.com/robbalian/rev. The recipe descends from Kev by Jared Palmer.