Tinyllama-pneumonia

TinyLlama Fine-tuned for Pneumonia Q&A

Model Description

This model is a fine-tuned version of TinyLlama/TinyLlama-1.1B-Chat-v1.0 specifically trained for answering questions about pneumonia.

⚠️ Important: This model is in development/testing phase and should NOT be used for actual medical diagnosis or treatment decisions.

Performance Metrics

• ROUGE-1: 0.3915
• ROUGE-2: 0.1944
• BLEU: 0.1060
• Perplexity: 2.17
• Response Quality: 100% proper sentence endings, 80% medical term usage

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained(
    "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    torch_dtype=torch.float16,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "your-username/tinyllama-pneumonia-improved")
tokenizer = AutoTokenizer.from_pretrained("your-username/tinyllama-pneumonia-improved")

# Generate response
question = "What is pneumonia?"
input_text = f"### Question: {question}\n### Answer:"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=80, temperature=0.6)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

Training Details

• Base Model: TinyLlama-1.1B-Chat-v1.0
• Training Method: LoRA (Low-Rank Adaptation)
• Training Epochs: 5
• Learning Rate: 1e-4
• Batch Size: 2
• LoRA Rank: 16
• LoRA Alpha: 32

Limitations

• This model is designed for educational/research purposes only
• Performance metrics indicate "Fair" quality - suitable for development/testing
• Should not be used for medical diagnosis or treatment decisions
• May generate inaccurate or incomplete medical information

Ethical Considerations

• This model should not replace professional medical advice
• Responses should be verified with medical professionals
• Use only for educational or research purposes

Citation

If you use this model, please cite:

@misc{tinyllama-pneumonia-improved,
  author = {Your Name},
  title = {TinyLlama Fine-tuned for Pneumonia Q&A},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/your-username/tinyllama-pneumonia-improved}
}