biomed-gpt-small

Biomed-GPT-Small

Overview

Biomed-GPT-Small is a lightweight GPT-style transformer model trained from scratch on a small biomedical corpus for educational and research purposes.

This model demonstrates:

• End-to-end LLM training using PyTorch
• Custom SentencePiece tokenizer
• Token-level next-token prediction
• Hugging Face Model Hub integration via PyTorchModelHubMixin

The model was trained on a toy biomedical dataset and is intended for experimentation, learning, and portfolio demonstration.

---

Architecture

• Transformer-based GPT model
• Context length: 32 tokens
• Embedding dimension: 128
• Layers: 4
• Heads: 4
• Vocabulary size: 128
• Framework: PyTorch

---

Training Data

The model was trained on a small curated biomedical text corpus for demonstration purposes.

It has not been trained on full PubMed or clinical datasets.

---

Intended Use

This model is intended for:

• Educational purposes
• LLM architecture experimentation
• Biomedical NLP prototyping

---

Limitations

• Very small dataset
• Strong overfitting expected
• Not suitable for real-world biomedical tasks
• No evaluation benchmark performed

---

⚠ Medical Disclaimer

This model is not validated for clinical use.

It must not be used for:

• Diagnosis
• Treatment recommendations
• Clinical decision making
• Patient care

---

Future Work

• Train on large-scale PubMed abstracts
• Increase vocabulary size
• Increase context length
• Evaluate perplexity on biomedical benchmarks
• Compare against BioGPT / PubMedGPT

---

Author

Oscar Yanez
Biomedical AI & LLM Engineering