adaption_llama_3_3_70b

Model Summary

adaption_llama_3_3_70b_instru_adapted_medqa_usmle_in_k_82b0c1ba is a supervised fine-tuned (SFT) medical language model based on Meta Llama 3.3 70B Instruct. The model was adapted using Low-Rank Adaptation (LoRA) on a multilingual medical instruction dataset consisting of clinical vignettes in English and Kazakh.

The model is designed to improve instruction following, clinical reasoning, and medical question answering. Training examples include enhanced prompts, high-quality completions, and reasoning traces, enabling the model to generate more structured and informative medical responses.

---

Usage

This model can be used for:

• Medical question answering
• Clinical vignette analysis
• Medical instruction following
• Clinical reasoning assistance
• Multilingual medical NLP research

Input

Natural language medical instructions or clinical case descriptions.

Output

Structured medical responses including diagnostic reasoning and treatment recommendations.

Known Limitations

• The model should not be used as a substitute for professional medical judgment.
• Outputs may contain factual inaccuracies or hallucinations.
• Performance may decrease on medical domains not represented in the training data.
• The model is intended for research and educational purposes.

---

System

This is a standalone causal language model.

Input requirements:

• Plain text prompts
• Clinical vignettes
• Medical questions
• Instruction-following tasks

The generated outputs may be integrated into downstream applications such as:

• Medical assistants
• Clinical education tools
• Question answering systems
• Medical NLP pipelines

---

Implementation Requirements

Training Hardware

The model was fine-tuned using the Adaption platform on GPU-based infrastructure.

Software

• PyTorch
• Hugging Face Transformers
• PEFT (LoRA)
• Supervised Fine-Tuning (SFT)

Compute Requirements

Training configuration:

• Base model: 70B parameters
• Training method: Supervised Fine-Tuning (SFT)
• LoRA adaptation
• Chat-formatted dataset

Inference requires multiple high-memory GPUs or equivalent accelerated hardware capable of serving a 70B parameter model.

---

Model Characteristics

Model Initialization

The model was initialized from:

meta-llama/Llama-3.3-70B-Instruct-Reference

It was fine-tuned using LoRA rather than trained from scratch.

---

Model Stats

---

Other Details

• LoRA adaptation on all linear layers
• No model pruning
• No quantization applied during training
• No differential privacy techniques were used

---

Data Overview

The model was trained on a multilingual medical instruction dataset containing 20,352 clinical examples.

The dataset contains approximately:

• 50% English
• 50% Kazakh

Each sample contains:

• question
• answer
• enhanced_prompt
• enhanced_completion
• reasoning_trace
• answer_idx
• meta_info
• options

---

Training Data

Training data consists of clinical vignettes covering multiple medical specialties including:

• Internal Medicine
• Pediatrics
• Psychiatry
• Cardiology
• Neurology
• Endocrinology
• Emergency Medicine
• Surgery
• Infectious Diseases
• Obstetrics and Gynecology

The data were converted into chat format and enhanced through prompt engineering to improve instruction-following behavior and reasoning quality.

---

Demographic Groups

The dataset contains clinical scenarios involving patients of different ages, sexes, and medical conditions. It does not contain personally identifiable information and is intended for educational and research purposes.

---

Evaluation Data

The model was evaluated during training using periodic validation checkpoints.

Training configuration:

• Epochs: 3
• Evaluation checkpoints: 5

No separate public benchmark results are included with this release.

---

Evaluation Results

Summary

The fine-tuned model demonstrated improved instruction following and more structured clinical reasoning compared with the base model on internal validation data.

---

Subgroup Evaluation Results

No formal subgroup evaluation was conducted.

Performance may vary across:

• Medical specialties
• Languages
• Question complexity

Further evaluation is recommended before deployment in real-world clinical environments.

---

Fairness

No dedicated fairness evaluation was performed.

The bilingual training data were designed to improve performance across both English and Kazakh medical questions, but performance disparities may still exist.

---

Usage Limitations

This model is intended for:

• Research
• Education
• Benchmarking
• Medical NLP development

It should not be used as the sole basis for clinical decision-making, diagnosis, or treatment.

Users should independently verify all medical recommendations.

---

Ethics

The model was developed for research into multilingual medical language models.

Potential risks include:

• Hallucinated medical information
• Incorrect clinical recommendations
• Misinterpretation of ambiguous cases

Users should ensure that outputs are reviewed by qualified healthcare professionals before being applied in clinical settings.