Llama-3-8B-Instruct-BotMed-v1-adapter

BotMed/Llama-3-8B-Instruct-BotMed-v1-adapter

<!-- Provide a quick summary of what the model is/does. -->

BotMed-v1-adapter is the LoRA adapter weights for a fine-tuned version of Llama 3 8B Instruct, adapted for medical chatbot use cases. This repository contains only the adapter weights, not the merged model — see Llama-3-8B-Instruct-BotMed-v1 for the full merged model.

Model Description

<!-- Provide a longer summary of what this model is. -->

BotMed-v1-adapter contains the LoRA (Low-Rank Adaptation) adapter weights fine-tuned on top of Meta's Llama 3 8B Instruct, specialized for medical chatbot and question-answering use cases. It was trained on the ruslanmv/ai-medical-chatbot dataset, with the base model loaded in 4-bit NF4 quantization to enable efficient fine-tuning on Kaggle GPUs.

The LoRA adapters (rank 8, alpha 16) target the attention and MLP projection layers (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj), allowing the model to adapt to medical dialogue patterns while preserving the general instruction-following capabilities of the base Llama 3 model. Training was run for 2 configured epochs with early stopping, using a cosine learning rate schedule and the paged AdamW 8-bit optimizer.

This adapter is the LoRA-only counterpart of Llama-3-8B-Instruct-BotMed-v1, released as a lightweight alternative for anyone who wants to apply the fine-tune on top of the official base model themselves rather than downloading the full merged weights. It is not a certified diagnostic tool and should not be used as a substitute for professional medical advice.

• Developed by: Bindupautra Jyotibrat (BJyotibrat)
• Shared by: BJyotibrat
• Model type: LoRA adapter for a causal language model
• Language(s) (NLP): English
• License: Llama 3 Community License
• Finetuned from model: meta-llama/Meta-Llama-3-8B-Instruct

Model Sources

<!-- Provide the basic links for the model. -->

• Repository: GitHub Repository
• Demo: BotMed Official Website

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

Intended to be loaded on top of meta-llama/Meta-Llama-3-8B-Instruct via PEFT for medical chatbot / medical question-answering conversational use cases.

Downstream Use

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

As the adapter-only counterpart of the first fine-tuning run in the BotMed project, this repo is best suited to exploratory and internal downstream uses rather than production deployment. Potential applications include:

(1) lightweight distribution — applying the fine-tune on top of the base model without redownloading the full ~16GB merged weights,

(2) baseline comparison, serving as a reference point against later BotMed adapter versions (e.g. v2) to evaluate the impact of changes in LoRA configuration, epochs, or dataset handling,

(3) further fine-tuning or continued LoRA training as a starting checkpoint for downstream medical NLP experiments, and

(4) merging into custom quantization/precision setups, since adapter-only repos give more flexibility over how the base model is loaded than a pre-merged model.

(Note: This adapter is built for Llama and distributed under the Llama 3 Community License. Any downstream applications, derivative models, or software integrating this adapter must comply with Meta's Acceptable Use Policy and include the required "Built with Llama" attribution.)

Given the lack of formal evaluation for this version (see Bias, Risks, and Limitations), it is not recommended for downstream use cases that reach real end-users without further validation — prefer v2 for those.

Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

Not a certified medical diagnostic tool. Unsafe and inappropriate for end-users seeking actual medical advice, diagnosis, or treatment — a licensed human medical professional should be the only resource used for that. Not intended to be used standalone; requires the base model meta-llama/Meta-Llama-3-8B-Instruct to function. Any use must comply with the Llama 3 Acceptable Use Policy.

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

• Fabricated attribution: Because the training data (ruslanmv/ai-medical-chatbot) consists of real doctor-patient Q&A transcripts, the model has learned to sign off responses with fabricated doctor names, specialties, and closing remarks (e.g. "Regards, Dr. Shinas Hussain, General & Family Physician"). These are stylistic artifacts inherited from training data, not real credentialed input, and should never be presented to end-users as if they came from an actual licensed physician.
• No formal evaluation: This v1 release has not been scored against held-out reference answers using metrics such as F1, ROUGE, Cosine Similarity, or BERTScore. Its quality has only been assessed qualitatively via example prompts.
• No clinical validation: The model has not been evaluated by medical professionals or validated against clinical guidelines.
• Dataset-driven bias: The model's knowledge and tone are shaped entirely by the ruslanmv/ai-medical-chatbot dataset, which may not reflect current medical guidelines and may skew toward specific conditions/specialties represented in that dataset.
• Hallucination risk: As with any LLM, the model can generate plausible-sounding but incorrect or incomplete medical information.
• Base model dependency: Results depend on correctly loading meta-llama/Meta-Llama-3-8B-Instruct as the base — using a different base model or precision setup than the one used in training may degrade output quality.

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases, and limitations of the model. In particular:

• Do not present model-generated doctor names, credentials, or signatures as genuine.
• Always pair model output with human review, especially in any workflow that reaches an end-user seeking health guidance.
• Treat outputs as a drafting/assistive aid, not a source of medical truth.
• Given the lack of formal evaluation for this version, prefer v2 for anything beyond experimentation, where possible.
• Ensure the correct base model (meta-llama/Meta-Llama-3-8B-Instruct) is loaded before applying this adapter.

How to Get Started with the Model

Use the code below to get started with the model.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_model_id = "meta-llama/Meta-Llama-3-8B-Instruct"
adapter_id = "BJyotibrat/Llama-3-8B-Instruct-BotMed-v1-adapter"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)

base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=bnb_config,
    device_map="auto",
    dtype=torch.float16,
)

model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

prompts = [
    "I have frequent fever and pain in chest"
]

for idx, t in enumerate(prompts, start=1):
    print(f"\nPrompt {idx}: {t}\n")

    messages = [{"role": "user", "content": t}]
    prompt = tokenizer.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )

    inputs = tokenizer(
        prompt,
        return_tensors="pt",
        truncation=True,
        max_length=2048
    ).to("cuda")

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=512,
            do_sample=True,
            temperature=0.7,
            top_p=0.9,
            repetition_penalty=1.2,
            pad_token_id=tokenizer.eos_token_id
        )

    input_length = inputs["input_ids"].shape[1]
    answer = tokenizer.decode(
        outputs[0][input_length:],
        skip_special_tokens=True,
        clean_up_tokenization_spaces=False
    ).strip()

    print(f"Answer {idx}: {answer}\n")

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

Fine-tuned on the ruslanmv/ai-medical-chatbot dataset.

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Fine-tuned using LoRA (Low-Rank Adaptation) on Kaggle GPUs, with the base model loaded in 4-bit quantization via BitsAndBytes. This adapter is the raw output of the LoRA fine-tuning run before merging into the base model (see Llama-3-8B-Instruct-BotMed-v1 for the merged version).

Preprocessing

Quantization (BitsAndBytes):

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    quantization_config=bnb_config,
    device_map="auto",
    attn_implementation=attn_implementation,
    dtype=torch.float16,
)

model = prepare_model_for_kbit_training(model)

LoRA Configuration:

peft_config = LoraConfig(
    r=8,
    lora_alpha=16,
    lora_dropout=0.1,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
)

model = get_peft_model(model, peft_config)
model.print_trainable_parameters()

Training Hyperparameters

• Training regime: fp16=False, bf16=False (mixed-precision compute via bnb_4bit_compute_dtype=torch.float16)
• Batch size: 2 per device (train and eval), with gradient accumulation steps of 4
• Optimizer: paged_adamw_8bit
• Epochs: 2 configured, with early stopping (training was manually stopped once eval loss stopped improving, ending at ~1.52 epochs)
• Evaluation/Save strategy: every 200 steps
• Logging: every 50 steps
• Warmup steps: 36
• Learning rate: 1e-4, cosine scheduler
• Gradient checkpointing: enabled
• Checkpoints kept: 2 (save_total_limit=2)
• Experiment tracking: Weights & Biases (wandb)

training_arguments = SFTConfig(
    output_dir=new_model,

    per_device_train_batch_size=2,
    per_device_eval_batch_size=2,
    gradient_accumulation_steps=4,

    optim="paged_adamw_8bit",
    num_train_epochs=2,

    eval_strategy="steps",
    eval_steps=200,
    save_strategy="steps",
    save_steps=200,

    logging_steps=50,

    warmup_steps=36,
    learning_rate=1e-4,
    lr_scheduler_type="cosine",

    fp16=False,
    bf16=False,

    report_to="wandb",
    gradient_checkpointing=True,

    load_best_model_at_end=True,
    metric_for_best_model="eval_loss",
    greater_is_better=False,

    save_total_limit=2,
)

Speeds, Sizes, Times

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

Final logged Weights & Biases metrics (last recorded checkpoint, global step 1800):

Full run history is available as a CSV export: wandb_run_history - BotMed v1.csv.

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Data, Factors & Metrics

Testing Data

<!-- This should link to a Dataset Card if possible. -->

No formal held-out evaluation set has been scored for this version.

Results

No formal evaluation metrics (F1, ROUGE, Cosine Similarity, BERTScore) have been computed for this version.

Example Outputs

Prompt: I have frequent fever and pain in chest

Answer: Hello, I have gone through your query. From what you have mentioned, it seems that you are having a dry cough. I would suggest you to use some cough syrups like Tab Robitussin or Tab Mucodyne for 3-5 days. Also, you can use some expectorant like Tab Mucopain for 3-5 days. If you have a fever, then you can use some antipyretic like Tab Paracetamol for 3-5 days.

(Note: raw model outputs include fabricated sign-offs such as doctor names and specialties)

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: Kaggle GPUs
• Hours used: 6 hours
• Cloud Provider: Kaggle

Technical Specifications

Model Architecture and Objective

LoRA adapter (rank 8, alpha 16) for the Llama 3 8B Instruct architecture, targeting the q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj modules, for medical chatbot dialogue generation. Trained with the base model loaded in 4-bit NF4 quantization.

Compute Infrastructure

Kaggle GPUs

Hardware

Kaggle GPU instance(s).

Software

🤗 Transformers, PEFT (LoRA), TRL (SFTConfig/SFTTrainer), BitsAndBytes, Weights & Biases and Kaggle.

Authors

Bindupautra Jyotibrat (BJyotibrat)

Contact

Email: [email protected]