Citrus1.0-llama-70B

Citrus: Leveraging Expert Cognitive Pathways in a Medical Language Model for Advanced Medical Decision Support

<p align="center"> <a href="https://arxiv.org/abs/2502.18274" target="_blank">📑Paper</a> ｜<a href="https://jdh-algo.github.io/Citrus/" target="_blank">🤗Github Page</a> ｜<a href="https://huggingface.co/jdh-algo/Citrus1.0-llama-70B" target="_blank">🤗Model</a> ｜<a href="https://huggingface.co/datasets/jdh-algo/Citrus_S3" target="_blank">📚Medical Reasoning Data</a> ｜ <a href="https://huggingface.co/datasets/jdh-algo/JMED" target="_blank">📚Evaluation Data</a> </p>

The Introduction to Our Work

1. Main approaches

<div align="center"> <img src="https://raw.githubusercontent.com/jdh-algo/Citrus/main/static/images/figure4-1-2.png" alt="image" width="75%"/> </div>

2. Overview of training stages and training data pipeline

<div align="center"> <img src="https://raw.githubusercontent.com/jdh-algo/Citrus/main/static/images/figure4-2-1.png" width="75%"> </div> Citrus is a medical language model that bridges the gap between clinical expertise and AI reasoning by emulating the cognitive processes of medical experts. The model is trained on a large corpus of simulated expert disease reasoning data in sft-stage-3, synthesized using a novel approach that accurately captures the decision-making pathways of clinicians.

The contributions of this work are as follows:

1. We propose a training-free reasoning approach that emulates the cognitive processes of medical experts, enabling large language models to enhance their medical capabilities in clinical diagnosis and treatment.

2. In conjunction with the data construction method, we introduce a multi-stage post-training approach to further improve the model’s medical performance.

3. We have made the Citrus model and its training data publicly available as open-source resources to advance research in AI-driven medical decision-making.

4. We have developed and open-sourced a large-scale, updatable clinical practice evaluation dataset based on real-world data, accurately reflecting the distribution of patients in real-world settings.

Notice

1. Our model is built with Llama3.1-70B, Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.
2. Our default license is MIT, provided it does not conflict with the Llama license.