tinyllama-medical-merged

TinyLlama-1.1B — Medical QLoRA (Merged)

A QLoRA fine-tuned variant of TinyLlama-1.1B-intermediate-step-1431k-3T, trained on medical question-answering data. The LoRA adapter has been merged into the base weights, so this repo is a standalone model — no separate adapter loading required.

⚠️ Intended use

This model was trained as part of a GenAI training lab to demonstrate the fine-tuning → merge → deploy workflow. It is not validated for real clinical or medical use and should not be used to inform actual medical decisions. Treat its outputs as educational examples of what a small fine-tuned LLM can produce, not medical advice.

Training details

• Base model: TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
• Method: QLoRA (4-bit NF4 quantization during training) via peft + trl.SFTTrainer
• LoRA config: r=8, lora_alpha=16
• Dataset: BI55/MedText — medical case/question-answer pairs
• Steps: 400
• Merge: adapter merged into the base model via PeftModel.merge_and_unload(), then saved as a full standalone model

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

repo_id = "tweijie/tinyllama-medical-merged"

model = AutoModelForCausalLM.from_pretrained(repo_id)
tokenizer = AutoTokenizer.from_pretrained(repo_id)

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, max_length=200)
prompt = "<s>[INST] A patient presents with restless leg sensations at night. What could be the cause? [/INST]"
print(pipe(prompt)[0]["generated_text"])


Limitations
1.1B parameters — small model, prone to hallucination and factual errors, especially on medical content.
Fine-tuned on a single narrow dataset (400 training steps); not broadly evaluated.
No safety/alignment tuning beyond the base model's own training.