rare-phenix-llama2-13b-raredis

RARE-PHENIX Module 1: Llama-2-13B RareDis Adapter

This repository contains a PEFT/LoRA adapter for RARE-PHENIX Module 1, the phenotype extraction module of RARE-PHENIX.

What this model does

This adapter is intended to extract rare disease phenotype mentions from clinical text.

Input:

The patient has global developmental delay and hypotonia.

Output:

The patient has <span class="condition">global developmental delay</span> and <span class="condition">hypotonia</span>.

Important scope note

This is Module 1 only.

It does not perform:

• HPO standardization
• HPO term ranking
• disease diagnosis
• gene or variant prioritization

The full RARE-PHENIX workflow includes additional downstream modules for HPO standardization and rare disease-relevant HPO prioritization.

Training data

This adapter was fine-tuned on the public RareDis corpus.

It is not the full Undiagnosed Diseases Network (UDN)-trained RARE-PHENIX model described in the manuscript. Controlled-access Undiagnosed Diseases Network (UDN) data and UDN-trained model artifacts are not included.

Base model

This is a LoRA adapter and requires access to the corresponding Meta Llama-2 base model:

meta-llama/Llama-2-13b-chat-hf

Users must separately request and obtain access to the gated Meta Llama-2 model through Hugging Face.

Example loading code

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "meta-llama/Llama-2-13b-chat-hf"
adapter_id = "shyrcathy/rare-phenix-llama2-13b-raredis"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)

base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    dtype=torch.float16,
    low_cpu_mem_usage=True,
    device_map="auto",
)

model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

Compute note

The 13B model requires more memory than the 7B model. For most users, start with:

shyrcathy/rare-phenix-llama2-7b-raredis

Use this 13B adapter if you have an appropriate high-memory GPU or Apple Silicon environment.

GitHub repository

Code and usage examples are available at:

https://github.com/cathyshyr/RARE_PHENIX_for_rare_disease_phenotyping

See:

docs/module1_hf_quickstart.md

Intended use

This adapter is intended for research use in rare disease phenotype extraction workflows.

Limitations

• This model is not intended for autonomous clinical diagnosis.
• Outputs should be reviewed by domain experts.
• The public adapter may perform differently from the UDN-trained RARE-PHENIX model described in the manuscript.
• The model may incorrectly tag negated findings, family history, or non-patient conditions; downstream post-processing and expert review are recommended.
• Do not send protected health information to environments that are not approved for PHI.

Citation

If you use this adapter, please cite:

Shyr, C., Hu, Y., Tinker, R.J., Cassini, T.A., Byram, K.W., Hamid, R., Fabbri, D.V., Wright, A., Peterson, J.F., Bastarache, L., and Xu, H. 2026. An artificial intelligence framework for end-to-end rare disease phenotyping from clinical notes using large language models. arXiv preprint arXiv:2602.20324.