Llama3.2-3b-Instruct-ClinicalSurgery

Uploaded model

• Developed by: vishal042002
• License: apache-2.0
• Finetuned from model : unsloth/llama-3.2-3b-instruct-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>

The model was trained on a custom dataset containing clinical surgery Q&A pairs. The dataset was compiled from: Open-source medical books

RUNNING THE MODEL THROUGH ADAPTER MERGE:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model_name = "unsloth/Llama-3.2-3B-Instruct"  
base_model = AutoModelForCausalLM.from_pretrained(base_model_name, torch_dtype=torch.float16, device_map="auto")

adapter_path = "vishal042002/Llama3.2-3b-Instruct-ClinicalSurgery"
base_model = PeftModel.from_pretrained(base_model, adapter_path)

tokenizer = AutoTokenizer.from_pretrained(base_model_name)

device = "cuda" if torch.cuda.is_available() else "cpu"
base_model.to(device)

# Sample usage
input_text = "What is the mortality rate for patients requiring surgical intervention who were unstable preoperatively?"
inputs = tokenizer(input_text, return_tensors="pt").to(device)

outputs = base_model.generate(**inputs, max_new_tokens=200, temperature=1.5, top_p=0.9)
decoded_output = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(decoded_output)

LOADING THE MODEL DIRECTLY:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "vishal042002/Llama3.2-3b-Instruct-ClinicalSurgery"
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained(model_name)

This model is designed to:

Answer questions about clinical surgery procedures. Provide information about surgical interventions.

Limitations:

The model should not be used as a substitute for professional medical advice. Responses should be verified by qualified medical professionals.