Downloads · 30 days
0
microgpt
gpahal/microgpt
microgpt is a machine learning model from gpahal. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
A micro GPT implementation and training pipeline in PyTorch. I built this to understand a few things:
Access
Public
Updated Jul 24, 2025
Repo size
1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt1 GB · 98%
At a glance
- License
- mit
- Access
- Public
- Created
- Jul 24, 2025
- Updated
- Jul 24, 2025
- SHA
- baa2ef6c
Model card
From the Hugging Face model README
A micro GPT implementation and training pipeline in PyTorch. I built this to understand a few things:
- Basics of attention and RoPE
- Training a GPT-like model with multiple GPUs including checkpointing and other considerations to make the run successful
- Multi phase training including combining/souping the model weights for the second stage with 3 runs of smaller amounts of high quality data
This repository contains the pretrained model and tokenizer.
See gpahal/microgpt for instructions on how to use the model.
- License
- mit
- Languages
- en
- Created
- Jul 24, 2025
- Updated
- Jul 24, 2025