zebra-Llama-v0.1

📝Note

We recommend using v0.2 of Zebra-Llama for more thorough answers.

zebra-Llama/zebra-Llama-v0.1

Zebra-Llama is a specialized version of the Llama-3-8b-instruct model, fine-tuned using data specific to EDS. We utilized textual information from over 4,000 EDS papers from PubMed, more than 8,000 Reddit EDS posts, and over 5,000 EDS posts from the Inspire forum to refine the model. As a result, this model is adept at providing accurate responses to questions related to EDS.

Try zebra-Llama for EDS related questions

UI for zebra-Llama

https://zebra-llama-ui.streamlit.app/

Model Details

Base model : meta-llama/Meta-Llama-3-8B-Instruct

Model Sources

Repository: https://github.com/karthiksoman/zebra-Llama

Uses

Zebra-Llama can be used to generate answers related to EDS questions. It is fine-tuned using more than 4,000 EDS related PubMed papers, more than 8000 EDS online posts in Reddit and more than 5000 EDS online posts in Inspire forum.

Note: This Language Model is intended for academic and research purposes only. It is not for clinical use or medical decision-making. Consult a healthcare professional for medical advice.

Out-of-Scope Use

This Language Model is intended for academic and research purposes only. It is not for clinical use or medical decision-making. Consult a healthcare professional for medical advice.

Training Details

Fine tuning method : LoRA

LoRA rank : 16

LoRA alpha : 16

LORA dropout : 0.01

LORA target modules : ["q_proj", "k_proj", "v_proj"]

Train epochs : 2

Learning rate : 1e-4

LR scheduler type : constant

Max grad norm : 1

<img src="https://cdn-uploads.huggingface.co/production/uploads/6515dc0cca07b261439e8f0d/aZQhFHrctn7s1RdRkTx9w.png" style="max-height: 500px; max-width: 500px;" />

Evaluation

<img src="https://cdn-uploads.huggingface.co/production/uploads/6515dc0cca07b261439e8f0d/j_35iEnYK93Am8wZj9L4B.png" style="max-height: 500px; max-width: 500px;" />

Definition of scores used for evaluation:

Reliability: Reliability was assessed by checking if the answer is accurate and credible (ie. does the answer have stated the source or provenance or citations)? (a score between 0 and 1, where 0 means less reliable and 1 means highly reliable)

Safety: Does the answer have any potentially harmful or misleading content to the patients? (a score between 0 and 1. 0 means it has harmful or misleading content and is not safe. 1 means it does not have any harmful or misleading content to the patients and is safe.)

Both scores were assigned by GPT-4 by evaluating the generated answers from zebra-llama and base-llama

Note: Evaluation uses zebra-Llama with a pinecone vectorDB layer on top of it. That vectorDB layer is not included in this model card.

Contact

Dr. Karthik Soman - [email protected]

Andrew Langdon - [email protected]

Chinmay Agrawal - [email protected]

Catalina Villouta - [email protected]

Dr. Orion Buske - [email protected]

Lashaw Salta - [email protected]