llama-3.1-8b-instruct-lora-clinical_iter1

RAG llama LoRA-1:-

This model is a PEFT/LoRA adapter single epoch fine-tuned on top of meta-llama/Llama-3.1-8B-Instruct. Fine tuning has been done on iter1 anchor-context-groundTruth_junkHandled dataset related to clinical trials. This fine tuned adapter on base LLM can be to address clinical trials related questions in general(macro/broad overview, patient eligiblity, operations etc related question). Further this can also be used for prompt engineering based on CT, or RAG generation(related to CT) based on retrieved chunks.

<!-- Provide a quick summary of what the model is/does. -->

Model Details:-

Model Description:-

• Model Type: Causal Large Language Model (Autoregressive Decoder),
• Base model: meta-llama/Llama-3.1-8B-Instruct,
• Maximum Sequence Length: 131,072 tokens (128K context window),
• Output Dimensionality (Hidden Size): 4096 dimensions,
• Supported Modality: Text,
• Training Dataset: iter2 anchor-context-groundTruth_junkHandled dataset (mapped via Chat Templates),
• Language: en,
• License: llama3.1 (Subject to Meta's Llama 3.1 Community License Agreement)

Model Sources:-

• Documentation:-Transformers Documentation / PEFT Documentation,
• Repository: Transformers on GitHub,
• Hugging Face: PEFT Adapters Hub

Fine tune model archtitecture:-

PeftModelForCausalLM(
  (base_model): LoraModel(
    (model): LlamaForCausalLM(
      (model): LlamaModel(
        (embed_tokens): Embedding(128256, 4096)
        (layers): ModuleList(
          (0-31): 32 x LlamaDecoderLayer(
            (self_attn): LlamaAttention(
              (q_proj): lora.Linear4bit(
                (base_layer): Linear4bit(in_features=4096, out_features=4096, bias=False)
                (lora_dropout): ModuleDict(
                  (default): Dropout(p=0.05, inplace=False)
                )
                (lora_A): ModuleDict(
                  (default): Linear(in_features=4096, out_features=16, bias=False)
                )
                (lora_B): ModuleDict(
                  (default): Linear(in_features=16, out_features=4096, bias=False)
                )
                (lora_embedding_A): ParameterDict()
                (lora_embedding_B): ParameterDict()
                (lora_magnitude_vector): ModuleDict()
              )
              (k_proj): lora.Linear4bit(
                (base_layer): Linear4bit(in_features=4096, out_features=1024, bias=False)
                (lora_dropout): ModuleDict(
                  (default): Dropout(p=0.05, inplace=False)
                )
                (lora_A): ModuleDict(
                  (default): Linear(in_features=4096, out_features=16, bias=False)
                )
                (lora_B): ModuleDict(
                  (default): Linear(in_features=16, out_features=1024, bias=False)
                )
                (lora_embedding_A): ParameterDict()
                (lora_embedding_B): ParameterDict()
                (lora_magnitude_vector): ModuleDict()
              )
              (v_proj): lora.Linear4bit(
                (base_layer): Linear4bit(in_features=4096, out_features=1024, bias=False)
                (lora_dropout): ModuleDict(
                  (default): Dropout(p=0.05, inplace=False)
                )
                (lora_A): ModuleDict(
                  (default): Linear(in_features=4096, out_features=16, bias=False)
                )
                (lora_B): ModuleDict(
                  (default): Linear(in_features=16, out_features=1024, bias=False)
                )
                (lora_embedding_A): ParameterDict()
                (lora_embedding_B): ParameterDict()
                (lora_magnitude_vector): ModuleDict()
              )
              (o_proj): lora.Linear4bit(
                (base_layer): Linear4bit(in_features=4096, out_features=4096, bias=False)
                (lora_dropout): ModuleDict(
                  (default): Dropout(p=0.05, inplace=False)
                )
                (lora_A): ModuleDict(
                  (default): Linear(in_features=4096, out_features=16, bias=False)
                )
                (lora_B): ModuleDict(
                  (default): Linear(in_features=16, out_features=4096, bias=False)
                )
                (lora_embedding_A): ParameterDict()
                (lora_embedding_B): ParameterDict()
                (lora_magnitude_vector): ModuleDict()
              )
            )
            (mlp): LlamaMLP(
              (gate_proj): lora.Linear4bit(
                (base_layer): Linear4bit(in_features=4096, out_features=14336, bias=False)
                (lora_dropout): ModuleDict(
                  (default): Dropout(p=0.05, inplace=False)
                )
                (lora_A): ModuleDict(
                  (default): Linear(in_features=4096, out_features=16, bias=False)
                )
                (lora_B): ModuleDict(
                  (default): Linear(in_features=16, out_features=14336, bias=False)
                )
                (lora_embedding_A): ParameterDict()
                (lora_embedding_B): ParameterDict()
                (lora_magnitude_vector): ModuleDict()
              )
              (up_proj): lora.Linear4bit(
                (base_layer): Linear4bit(in_features=4096, out_features=14336, bias=False)
                (lora_dropout): ModuleDict(
                  (default): Dropout(p=0.05, inplace=False)
                )
                (lora_A): ModuleDict(
                  (default): Linear(in_features=4096, out_features=16, bias=False)
                )
                (lora_B): ModuleDict(
                  (default): Linear(in_features=16, out_features=14336, bias=False)
                )
                (lora_embedding_A): ParameterDict()
                (lora_embedding_B): ParameterDict()
                (lora_magnitude_vector): ModuleDict()
              )
              (down_proj): lora.Linear4bit(
                (base_layer): Linear4bit(in_features=14336, out_features=4096, bias=False)
                (lora_dropout): ModuleDict(
                  (default): Dropout(p=0.05, inplace=False)
                )
                (lora_A): ModuleDict(
                  (default): Linear(in_features=14336, out_features=16, bias=False)
                )
                (lora_B): ModuleDict(
                  (default): Linear(in_features=16, out_features=4096, bias=False)
                )
                (lora_embedding_A): ParameterDict()
                (lora_embedding_B): ParameterDict()
                (lora_magnitude_vector): ModuleDict()
              )
              (act_fn): SiLUActivation()
            )
            (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
            (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
          )
        )
        (norm): LlamaRMSNorm((4096,), eps=1e-05)
        (rotary_emb): LlamaRotaryEmbedding()
      )
      (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
    )
  )
)

How to Load and Run Inference

You can load this model using bitsandbytes for 4-bit quantization and peft to apply the adapter weights. Tokenizer formatting utilizes the base model's official chat template to map query/question tokens alongside retrieved context blocks.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

MODEL_ID = "meta-llama/Llama-3.1-8B-Instruct"

# 1. Load the tokenizer from the base model
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)

# 2. Configure 4-bit quantization
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True
)

# 3. Load quantized base model
base_model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    quantization_config=bnb_config,
    device_map="auto",
    torch_dtype=torch.float16
)

# 4. Load your current LoRA fine-tuned adapter
model = PeftModel.from_pretrained(
    base_model,
    "vab46/llama-3.1-8b-instruct-lora-clinical_iter2_epoch2"
)
model.eval()

# ==========================================
# 5. STRUCTURE RAG PROMPTS WITH CHAT TEMPLATE
# ==========================================
# Mapping the query/question with relevant context chunks into the message schema
SYSTEM_PROMPT = "You are a helpful assistant specialized in clinical data analysis."
retrieved_context = "..."  # Paste or map your retrieved context blocks here
user_query = "..."         # Paste your question/anchor here

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {
        "role": "user", 
        "content": f"CONTEXT(RETRIEVED_BLOCKS):
CONTEXT:
TITLE: WATCH-STEP : Pilot Trial: Smartwatch-Guided Secondary Prevention After Stroke Randomized Trial of Nurse-led Program With Active vs Passive Smartwatch in Minor Stroke. A Randomized Controlled Trial Evaluating a Nurse-led Secondary Prevention and Physical Activity Program Supported by Either an Active Smartwatch (Structured Feedback) or Passive Smartwatch in Patients With Minor Stroke.
SUMMARY: After a first stroke or transient ischemic attack (TIA), the risk of recurrence is high in the weeks and months following the initial event. There are several modifiable risk factors that can reduce this risk, such as blood pressure, diet, physical activity, and smoking. Many stroke patients (NIHSS \< 5) have a low daily step count during the early recovery period, despite a good functional prognosis.
Active smartwatches provide real-time feedback, track progress, and set personalized walking goals, thereby boosting motivation and adherence to physical activity recommendations.
The combination of advice provided by nurses and active behavioral coaching supported by a smartwatch, compared to passive monitoring, could significantly increase daily step counts over a 12-week period. The results of this research will help guide future large-scale secondary prevention strategies integrating digital health and structured nursing support.
INCLUSION_CRITERIA:
Age ≤ 40-75
Patients with recent minor ischemic stroke or TIA, initial NIHSS ≤5 (\< 30 days)
mRS ≤2 before stroke and on discharge
Patient with Wifi access (for data transmission)
Able to understand smartwatch instructions
Written informed consent
Ambulatory ≥10 meters without assistance and without walking aid
Expected to live at home during the study period.

QUESTION:
What is the main goal of this clinical trial regarding secondary prevention after a minor stroke?"
    }
]

# Apply Llama 3.1 specific formatting tokens automatically
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

# Generate response
with torch.no_grad():
    outputs = model.generate(
        inputs, 
        max_new_tokens=512, 
        eos_token_id=tokenizer.eos_token_id
    )

response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
print("RELEVANT_ANSWER(generated by LLM):", response)

<u>Evaluation(on clinical trials test_data splitted in 0.1 ratio at seed=42)</u>:-

STRATIFIED RESULTS:

GROUNDEDNESS / COMPLETENESS DIAGNOSTICS:

Head to head base vs LoRA in Rouge-score & Bert-score:

OVERALL MACRO RESULTS:

Training Dataset:

• Samples

  | anchor | positive | reference_answer | | ----------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | What is the purpose of this clinical trial regarding the PhysioMaster smartphone application? | TITLE: Validity and Reliability of the PhysioMaster in Assessing Posture and Range of Motion in Patients With Stroke<br>SUMMARY: The smartphone application named PhysioMaster is an easy-to-use, accessible, and freely available application. Although it has been used in several previous studies, no validity or reliability study has yet been conducted in patients with stroke. Therefore, the aim of this study is to investigate the validity and reliability of the PhysioMaster smartphone application in patients with stroke.<br>Accordingly, our hypotheses are as follows:<br>H0: The PhysioMaster application is not a valid and reliable measurement tool for the assessment of posture and range of motion in patients with stroke.<br>H1: The PhysioMaster application is a valid and reliable measurement tool for the assessment of posture and range of motion in patients with stroke. Participants will not take any intervention.<br>Participants will be evaluated in terms of posture and range of motion by using PhysioMaster, Universal Goniometer and PostureScreen Mobile. During the first-day assessment, physiotherapists A and B will complete both the PhysioMaster application assessments and all other evaluations on the same patient without seeing each other's assessment scores. In order to evaluate the test-retest reliability of the application, physiotherapist A will repeat the assessments on Physiomaster on the same patient after 3 days.<br>INCLUSION_CRITERIA:<br>Presence of hemiplegia caused solely by stroke<br>Being older than 40 years and younger than 70 years of age<br>At least 6 months have passed since the stroke<br>Ability to maintain independent standing for at least 1 minute<br>Being at stage 4 or higher according to the Brunnstrom stages of stroke recovery<br>Ability to understand and follow instructions<br>Voluntary agreement to participate in the study | The purpose of this clinical trial is to investigate the validity and reliability of the PhysioMaster smartphone application in patients with stroke. | | What is the primary mechanism by which kTMP-enhanced motor rehabilitation aims to improve arm/hand function in patients with chronic stroke recovery? | TITLE: kTMP-Enhanced Motor Rehabilitation for Chronic Stroke Recovery<br>SUMMARY: kTMP, kilohertz transcutaneous magnetic perturbations, is a low intensity transcranial magnetic stimulation technique that will be used in this study to promote arm/hand rehabilitation in patients who have been disabled by stroke.<br>INCLUSION_CRITERIA: 1. Adults aged 18-80 years who have a history of an ischemic or hemorrhagic stroke ≥ 12 m prior to enrollment.<br>2. Fugl-Meyer Assessment: Upper Extremity (FMA UE) score 20-50.<br>3. Ability to communicate, understand, and give appropriate consent. Participants can follow two-step commands.<br>4. Motor evoked potentials in a hand muscle when stimulating ipsilesional cortex with suprathreshold single-pulse TMS (MEP positive). | kTMP-enhanced motor rehabilitation aims to improve arm/hand function by using low intensity transcranial magnetic stimulation, specifically kilohertz transcutaneous magnetic perturbations, to promote recovery. | | What is the main goal of this clinical trial regarding secondary prevention after a minor stroke? | TITLE: WATCH-STEP : Pilot Trial: Smartwatch-Guided Secondary Prevention After Stroke Randomized Trial of Nurse-led Program With Active vs Passive Smartwatch in Minor Stroke. A Randomized Controlled Trial Evaluating a Nurse-led Secondary Prevention and Physical Activity Program Supported by Either an Active Smartwatch (Structured Feedback) or Passive Smartwatch in Patients With Minor Stroke.<br>SUMMARY: After a first stroke or transient ischemic attack (TIA), the risk of recurrence is high in the weeks and months following the initial event. There are several modifiable risk factors that can reduce this risk, such as blood pressure, diet, physical activity, and smoking. Many stroke patients (NIHSS \< 5) have a low daily step count during the early recovery period, despite a good functional prognosis.<br>Active smartwatches provide real-time feedback, track progress, and set personalized walking goals, thereby boosting motivation and adherence to physical activity recommendations.<br>The combination of advice provided by nurses and active behavioral coaching supported by a smartwatch, compared to passive monitoring, could significantly increase daily step counts over a 12-week period. The results of this research will help guide future large-scale secondary prevention strategies integrating digital health and structured nursing support.<br>INCLUSION_CRITERIA:<br>Age ≤ 40-75<br>Patients with recent minor ischemic stroke or TIA, initial NIHSS ≤5 (\< 30 days)<br>mRS ≤2 before stroke and on discharge<br>Patient with Wifi access (for data transmission)<br>Able to understand smartwatch instructions<br>Written informed consent<br>Ambulatory ≥10 meters without assistance and without walking aid<br>Expected to live at home during the study period. | The main goal of this clinical trial is to evaluate the effectiveness of a nurse-led secondary prevention and physical activity program supported by either an active smartwatch (structured feedback) or passive smartwatch in patients with minor stroke, specifically focusing on increasing daily step counts over a 12-week period. |

Training Hyperparameters

The following configurations were used during training:

• Train Batch Size: 2
• gradient_accumulation_steps: 8,
• Total Epochs: 1
• Weight Decay: 0.01
• Learning Rate: 0.0002
• Output Directory: /kaggle/working/rag3_llama_lora
• LR Scheduler Type: SchedulerType.COSINE
• Warmup Steps: 22
• FP16 Precision: True
• BF16 Precision: False
• Gradient Checkpointing: True
• Logging Strategy: IntervalStrategy.STEPS
• Logging Steps: 25
• Logging First Step: True
• Save Strategy: SaveStrategy.STEPS
• Save Steps: 100
• **#save_strategy:, "epoch",#use this (ignore previous 2 stepwise checkpoint) if strong gpu +VRAM, else use stepwise ckpt
• Save Total Limit: 1
• Evaluation Strategy: IntervalStrategy.NO
• Seed: 42
• Report To: []
• Dataloader Num Workers: 2
• Dataloader Pin Memory: True
• Remove Unused Columns: False

Training Loss History:

NA

Training loss:-0.2150