Downloads · 30 days
0
spark-mini-gpt
LucasIlmonen/spark-mini-gpt
spark-mini-gpt is a machine learning model from LucasIlmonen. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
A small GPT model I built from scratch in PyTorch, trained on a short repeated story about a robot named Spark.
Access
Public
Updated Jul 15, 2026
Repo size
651 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin651 MB · 100%
At a glance
- Access
- Public
- Created
- Jul 14, 2026
- Updated
- Jul 15, 2026
- SHA
- bc42d86e
Model card
From the Hugging Face model README
Spark Mini-GPT
A small GPT model I built from scratch in PyTorch, trained on a short repeated story about a robot named Spark.
Architecture
- 12-layer transformer
- 768-dim embeddings, 12 attention heads
- GPT-2 tokenizer (50,257 vocab)
Training
- ~7,600 tokens (a short story repeated 50 times)
- 10 epochs with AdamW
- Final loss: ~0.007
Why I made this
This was a class project to learn how GPT models actually work
- Created
- Jul 14, 2026
- Updated
- Jul 15, 2026