ClinSeek-35B-A3B

ClinSeek-35B-A3B

ClinSeek-35B-A3B is the open-source model of ClinSeekAgent: Automating Multimodal Evidence Seeking for Agentic Clinical Reasoning. It is Qwen/Qwen3.5-35B-A3B fine-tuned on ClinSeekAgent trajectories generated by GPT-5.6 Sol.

In ClinSeekAgent, evidence is not handed to the model in a curated prompt. The agent receives the task, the patient identifier, and the prediction time, and retrieves patient-specific evidence from the raw EHR with tools before answering. ClinSeek-35B-A3B learns this multi-round evidence-seeking process: it reasons in its native thinking block and acts through native tool calls.

On AgentEHR-Bench, ClinSeek-35B-A3B reaches 35.6 average F1 (+16.1 over its base model), outperforming all evaluated open-source models, including Qwen3.5-397B-A17B, as well as Claude Opus 5, and reaching 92% of its teacher.

<p align="center"> <img src="assets/performance.png" alt="ClinSeek-35B-A3B performance on AgentEHR-Bench" width="92%"> </p>

Evaluation uses the questions and labels of AgentEHR-Bench but runs them through ClinSeekAgent's own tool interface and strict temporal runtime view, on a fixed sample of 100 questions for each of five tasks. The scores below may therefore differ from those reported in the AgentEHR paper and are best compared within this table.

Release information

The weights on main are the checkpoint reported in the revised paper (arXiv v2). The checkpoint released with arXiv v1 (May 2026) is kept under the v1 tag.

Training data

GPT-5.6 Sol runs ClinSeekAgent with the 16 EHR tools and the strict temporal policy on the 3,000 questions of the AgentEHR-Bench training split, which cover six tasks (diagnoses, procedures, laboratory tests, microbiology tests, prescriptions, and transfers). We keep the 2,947 trajectories that end with an answer and drop those longer than 52K tokens, leaving 2,681, split at random into 2,628 for training and 53 for validation. None of the evaluation questions appears in the training data.

During data generation the teacher records its reasoning with a think tool before each action. For training, this reasoning is moved into the student's native thinking block of the following turn, so the student reasons in its thinking channel instead of calling the tool. Tool calls are rendered in the Qwen3.5 native tool-call format. The loss is computed on assistant tokens only (reasoning, tool calls, and the final answer).

The trajectories are not released: they contain records from credentialed clinical datasets.

Training configuration

Evaluation

AgentEHR-Bench test split, 100 questions per task, F1 (%). All models run ClinSeekAgent with the same tools, temporal policy, and inference settings. The base and fine-tuned models are served locally with identical decoding settings.

Every gain over the base model is significant under a paired bootstrap (95% CI).

Compared with its base model on the same 500 questions, ClinSeek-35B-A3B improves both precision (18.6 → 32.4) and recall (31.1 → 53.3). It writes SQL with joins, aggregation, or time filters in 53% of the questions (base: 0.8%), reads 72% of the available tables (base: 36%), and reads imaging or discharge notes in 74% of the questions (base: 12%).

Usage

ClinSeek-35B-A3B is an agent model: it expects the ClinSeekAgent EHR tools and a multi-turn tool loop. Use the ClinSeekAgent repository, which provides the EHR MCP server, tool schemas, prompts, the AgentEHR-Bench and ClinSeek-Bench runners, and scoring.

Serving with vLLM

The paper's evaluation served the model with vLLM 0.28.1 as follows (scripts/run_vllm_server.sh in the repository applies the same flags):

vllm serve UCSC-VLAA/ClinSeek-35B-A3B \
  --served-model-name ClinSeek-35B-A3B \
  --max-model-len 196608 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --language-model-only

Inference settings used in the paper: thinking enabled with a 4,096-token budget, temperature 0, at most 8,192 output tokens per call, tool results truncated to 100,000 characters, and at most 200 rounds per question.

The model was trained with an EHR-only tool space (no web browsing). Do not add an instruction to call ehr.think: the model reasons in its thinking block instead.

Loading with transformers

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

model_id = "UCSC-VLAA/ClinSeek-35B-A3B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="auto"
)

The checkpoint keeps the Qwen3.5 architecture (Qwen3_5MoeForConditionalGeneration); ClinSeek-35B-A3B was trained and evaluated on text only.

Limitations and responsible use

• The model was trained on trajectories over MIMIC-IV-derived patient records (PhysioNet credentialed data). Users must comply with the PhysioNet data-use agreement and must not attempt to recover patient-level information.
• It was trained on AgentEHR-Bench tasks only; performance on other clinical tasks is not established.
• It is a research model, not a medical device. Its outputs must not be used for clinical diagnosis, treatment, triage, or patient management.

Acknowledgements

We thank Yusheng Liao and Chaoyi Wu for their work on EHR-R1 / EHR-Bench and AgentEHR-Bench, which this project builds on, and for their help in checking the data details of our benchmark and evaluation.

Citation

@misc{wu2026clinseekagent,
  title         = {ClinSeekAgent: Automating Multimodal Evidence Seeking for Agentic Clinical Reasoning},
  author        = {Wu, Juncheng and Zhang, Letian and Wang, Yuhan and Tu, Haoqin and Chen, Hardy and Wang, Zijun and Xie, Cihang and He, Yufan and Zhao, Can and Xu, Daguang and Tang, Yucheng and Zhou, Yuyin},
  year          = {2026},
  eprint        = {2605.20176},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2605.20176}
}

Please also cite AgentEHR-Bench, MIMIC-IV, and Qwen3.5 where applicable.