OphReason-Vision

OphReason-Vision Dataset

🔗 Open Source Links

• OphReason-Vision Dataset: Hugging Face | ModelScope
• OphVLM-R1 Model Weights: Hugging Face | ModelScope
• OphAgent System Code: GitHub
• Paper: WAICA 2026

📰 News

• 2026.07.07 🎉 Paper "OphVLM-R1: Efficient Ophthalmic Reasoning via Curriculum RL" accepted by WAICA 2026!
• 2025.11.28 📊 OphReason-Vision dataset subset open-sourced

Introduction

OphReason-Vision is a high-quality dataset constructed specifically for ophthalmic multimodal reasoning, containing 15,418 expert-verified Chain-of-Thought reasoning trajectories. Built from 100K+ clinical cases and 30+ public datasets, with automated quality control and expert-collaborative review achieving inter-rater agreement Cohen's $\kappa = 0.82$.

The currently open-sourced subset contains 3,418 samples for model cold-start supervised fine-tuning (Cold-Start SFT).

Dataset Construction

OphReason-Vision uses a three-stage closed-loop pipeline, addressing three deficiencies in existing ophthalmic multimodal training data: high heterogeneity across sources with incompatible formats, absence of structured reasoning chains where public datasets provide only image-level labels, and skewed difficulty distribution that overrepresents common conditions while underserving rare diseases.

Stage 1: Data Standardization

Integrates 100K+ clinical cases with 30+ public datasets using a dual-stream strategy:

• Text stream: Parses unstructured electronic medical records into standardized JSON, resolving inconsistent terminology through a curated ophthalmic synonym table
• Visual stream: Generates detailed textual descriptions for datasets that contain only image-level labels, enriching visual information available for reasoning synthesis

Stage 2: Structured Reasoning Synthesis

Uses Intern-S1 to generate multi-dimensional instructions covering lesion localization, multimodal diagnosis, and knowledge question answering. For each instruction, produces a Chain-of-Thought reasoning chain following the clinical diagnostic workflow: visual sign identification, knowledge retrieval, pathological analysis, and clinical decision.

Given input $x$, reasoning chain $z = (z_1, \ldots, z_n)$, and answer $y$, the chain probability is:

$$P(z|x) = \prod_{t=1}^{|z|} P(z_t | x, z_{<t})$$

Quality control employs an LVLM-as-a-Judge mechanism using Intern-S1 with threshold $\tau = 0.7$, determined via a pilot study on 500 expert-reviewed samples to maximize F1. The judge evaluates four dimensions: medical correctness, reasoning consistency, step completeness, and clarity.

Stage 3: Expert-Collaborative Optimization

Three board-certified ophthalmologists review the 18% of samples flagged as difficult, achieving inter-rater agreement Cohen's $\kappa = 0.82$. Disagreements are resolved through discussion until consensus.

Difficulty grading partitions the dataset by the base model's perplexity:

$$d(x, y) = 1 - P_\theta(y|x), \quad \text{Level}(x, y) = \begin{cases} \text{Easy}, & d < \tau_1 \ \text{Medium}, & \tau_1 \leq d < \tau_2 \ \text{Hard}, & d \geq \tau_2 \end{cases}$$

Dataset Statistics

The final dataset contains 15,418 records:

• Training set: 13,418 samples
• Evaluation set: 2,000 samples (held-out clinical cases with no shared patient IDs)

To mitigate data contamination with external benchmarks, three checks are performed: perceptual hashing to detect near-duplicate images, cross-referencing source dataset identifiers to exclude overlapping samples, and manual audit of shared source institutions.

The currently open-sourced subset contains 3,418 samples for cold-start supervised fine-tuning.

Data Quality Assurance

• Automated quality control: LVLM-as-a-Judge mechanism with threshold $\tau = 0.7$
• Expert-collaborative review: Three board-certified ophthalmologists review difficult samples
• Inter-rater agreement: Cohen's $\kappa = 0.82$
• Data contamination detection: Perceptual hashing, source identifier cross-referencing, manual audit

Usage Example

from datasets import load_dataset

# Load dataset
dataset = load_dataset("QiZishi/OphReason-Vision")

# View sample
print(dataset["train"][0])

Ethics Statement

OphReason-Vision draws from two data sources with appropriate ethical authorization:

• 30+ public datasets: Published research with original IRB approval, used in accordance with their respective licenses
• 100K+ clinical cases: Collected under IRB/Ethics Committee approval with research data use authorization

Comprehensive de-identification was performed, including removal of protected health information (PHI), metadata scrubbing, and facial cropping to ensure patient privacy. A waiver of informed consent was approved by the IRB for this retrospective research. All expert reviewers involved in data quality assessment are board-certified ophthalmologists who participated under institutional review protocols.

Citation

If you use OphReason-Vision, please cite our work:

@inproceedings{qi2026ophvlmr1,
  title={OphVLM-R1: Efficient Ophthalmic Reasoning via Curriculum RL},
  author={Qi, Zishi and Hu, Xiaoya and Pan, Huilin and Gao, Ang and Hou, Jiaxin and Li, Jiankun and Qian, Yongao},
  booktitle={Proceedings of the World Artificial Intelligence Conference (WAICA)},
  year={2026}
}

License

This dataset is licensed under CC-BY-NC-SA-4.0.

Related Resources

• OphVLM-R1 Model: Hugging Face | ModelScope
• OphAgent System: GitHub
• Paper: WAICA 2026