qwen35-extract-tsv-v2

qwen35-extract-tsv-v1

Selective full-FT of Qwen3.5-9B for medical admission-fact extraction from document page images. Output is compact TSV (CTX line + 6 TAB-separated columns/fact), not JSON.

Quality: ~98–100% recall / ~100% precision post-dedup (image-grounded judges). Speed: 4.4–4.9 s/page with the DFlash drafter on 2×H100 (see below).

Architecture: hybrid GatedDeltaNet (24 GDN + 8 full-attn) + vision tower + MTP head. The arch drives the gotchas below — read them before serving.

Serve (vLLM, recommended)

vllm serve alhparsa/qwen35-extract-tsv-v1 \
  --speculative-config '{"method":"dflash","model":"alhparsa/qwen35-9b-dflash","num_speculative_tokens":15}' \
  --max-model-len 32768 \
  --attention-backend flash_attn \
  --max-num-batched-tokens 32768 \
  --mm-processor-kwargs '{"max_pixels": 1003520}' \
  --default-chat-template-kwargs '{"enable_thinking": false}'

The --speculative-config is lossless speedup only (the drafter proposes tokens, this model verifies them — identical output). Drop it to serve ~2× slower with no other change. Run one replica per GPU and round-robin requests for best throughput.

Decode params — must match training

temperature 0.7   top_p 0.8   top_k 20   repetition_penalty 1.08   seed 0

Greedy / low-temp measurably hurts recall. Do not change these.

Message format

• System: the short system prompt (short_sys.txt) with {today_date} filled to today (ISO).
• User: [Available tags list] + "Previous context before this window:\n\n", then per page "Page N\n\n" + the page image, then "Extract all facts from these consecutive pages."
• The model only emits tags present in the Available tags list you pass in the user message.
• Patient identity (name/MRN/DOB/CSN) comes out on the CTX first line, not as fact rows.

Sample input / output

Real held-out example (an Occupational Therapy evaluation page, mid-document window). Page image elided.

Input messages:

[
  {"role": "system", "content": "% You are an AI specialized in extracting ... Today's date is: 2026-06-18\n% ... (short_sys.txt, full text — the 6-column TSV contract)"},
  {"role": "user", "content": [
    {"type": "text", "text": "Available tags:\nMDS Tags:\n- mds_a_id_info ... - mds_q_participation\nDocument Type Tags:\n- med_diag, allergy, vital, lab_result, evaluation, therapy, progress_note, care_plan, medication, physical_exam ...\n\nPrevious context before this window:\nActive Service Date: 2026-05-17; source_label: \"Today's Date\". This page continues the Occupational Therapy Evaluation. It includes AM-PAC Daily Activity scores, Transfers (deferred due to pain), Balance, Coordination, Functional Cognition (Intact/Alert), ROM, Strength ...\n\n"},
    {"type": "text", "text": "Page 40\n\n"},
    {"type": "image_url", "image_url": {"url": "data:image/png;base64,<...page...>"}},
    {"type": "text", "text": "Extract all facts from these consecutive pages."}
  ]}
]

Output (TSV — CTX line, then page⇥fact⇥tags⇥temporal_date⇥temporal_text⇥confidence):

CTX: Active Service Date: 2026-05-17; source_label: "Today's Date". This page concludes the Occupational Therapy Evaluation with short-term goals for transfers, dressing, and bed mobility, classifies the evaluation as high complexity, and includes ADL flowsheet data and a Last Skin Assessment dated 05/18/2026.
40	Favorable for Outcome: PLOF;Motivated;Intact Cognition	evaluation,therapy	2026-05-17	5/17/2026	-
40	Barriers to Outcome: Comorbidities;Lives Alone;Decreased Activity Tolerance;Decreased UE Fxnl Use;Pain	evaluation,therapy	2026-05-17	5/17/2026	-
40	Problem List: Decreased ADL status;Decreased UE strength;Decreased UE ROM;Decreased LE ROM;Decreased LE strength;Decreased activity tolerance;Decreased transfer ability;Decreased coordination;Impaired functional mobility;Impaired balance;Impaired gait;Pain;Decreased high-level ADLs	evaluation,therapy	2026-05-17	5/17/2026	-
40	Rehab Potential: Good	evaluation,therapy	2026-05-17	5/17/2026	-
40	OT Priority: 3 SAT 5/16; FBR	evaluation,therapy	2026-05-17	5/17/2026	-
40	OT Frequency: 1-2x/wk	evaluation,therapy	2026-05-17	5/17/2026	-

Notes: the Previous context block carries the prior window's CTX so facts spanning a page break aren't lost; page is the real source page (40); temporal_date (ISO) and temporal_text (as written) are populated from the active service date; tags come only from the Available tags list. Patient identity (when present) lands on the CTX line, not as fact rows.

Plain transformers (sanity check, no vLLM/drafter)

See hf_infer_sample.py in the training repo — loads via AutoModelForImageTextToText, max_pixels=1003520, the decode params above. No drafter (vLLM-only), no PDF render/windowing/dedup.

Serving gotchas (each cost a debug round)

1. Never enable ngram/prompt-lookup speculation — GatedDeltaNet state can't roll back rejected drafts → silent corruption. DFlash/MTP only.
2. Keep --mm-processor-kwargs '{"max_pixels": 1003520}' — without it images are processed at native res (off-distribution + slow). preprocessor_config.json is included here.
3. The 15 mtp.* tensors are required for speculative decode; they are present in this checkpoint. If you re-merge/requant, re-graft them or spec decode breaks.
4. Prefix caching auto-disables on hybrid models; attention block size is forced for mamba page alignment — expected, not a bug.
5. PyPI vLLM wheel may need CUDA-13 runtime libs on LD_LIBRARY_PATH.

Drafter

alhparsa/qwen35-9b-dflash is a mirror of public z-lab/Qwen3.5-9B-DFlash. It was trained on stock Qwen3.5-9B, not recalibrated to this fine-tune, so acceptance is a bit below z-lab's headline — still a clear win.

Recommended pipeline (prod recall)

Render PDF pages at 2.0 zoom → 3-page windows, stride 2 (1-page overlap) → concurrent requests round-robin across replicas → parse TSV → extract identity from CTX line → reseed-retry any window returning <5 facts → union → semantic dedup. Single-window inference under-recalls on multi-page docs.