Downloads · 30 days
0
pythia-160M-finetuned-MIMIC-III
mia-project-2025/pythia-160M-finetuned-MIMIC-III
pythia-160M-finetuned-MIMIC-III is a machine learning model from mia-project-2025. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This repository contains a Pythia-160M model fine-tuned on the MIMIC-III dataset for clinical text generation. The model was trained to predict the impressions field from patient notes.
Access
Public
Updated Aug 30, 2025
Repo size
3.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt2.6 GB · 57%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Aug 30, 2025
- Updated
- Aug 30, 2025
- SHA
- 10d48947
Model card
From the Hugging Face model README
Pythia-finetuned-Mimic-III
This repository contains a Pythia-160M model fine-tuned on the MIMIC-III dataset for clinical text generation. The model was trained to predict the impressions field from patient notes.
Model Details
- Base model:
EleutherAI/pythia-160m - Dataset:
hejazizo/mimic-iii - Task: Causal language modeling (predict impressions from clinical notes)
- Tokenizer: AutoTokenizer from
EleutherAI/pythia-160m - Maximum sequence length: 512
- Block size for LM: 1024
- Fine-tuning strategy: Full fine-tuning with
Trainer - Precision: FP16 (mixed precision)
- Batch size: 8 per device, gradient accumulation steps 2
- Learning rate: 1e-5
- Epochs: 10
- Weight decay: 0.01
- Early stopping: Patience = 2 epochs
Evaluation metrics at epoch 10:
- eval_loss: 2.7841
- eval_perplexity: 16.17
- eval_runtime: 35.3012
- eval_samples_per_second: 184.9230
- eval_steps_per_second: 5.7790
- epoch: 10.0000
- License
- apache-2.0
- Created
- Aug 30, 2025
- Updated
- Aug 30, 2025