Downloads · 30 days
63
80% of all-time downloads
tinystories-minigpt
kryln/tinystories-minigpt
tinystories-minigpt is a machine learning model from kryln. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
A ~5M parameter character-level GPT model trained from scratch on TinyStories.
All-time downloads
79
Public
Parameters
4.9M
19.6 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors19.6 MB · 100%
At a glance
- License
- mit
- Model type
- minigpt
- Access
- Public
- Created
- Jul 27, 2026
- Updated
- Jul 27, 2026
- SHA
- 17e48e4e
Datasets
Model card
From the Hugging Face model README
MiniGPT - TinyStories
A ~5M parameter character-level GPT model trained from scratch on TinyStories.
Model Details
| Parameter | Value |
|---|---|
| Parameters | ~4.9M |
| Vocab Size | 72 (character-level) |
| Embedding Dim | 256 |
| Layers | 6 |
| Attention Heads | 8 |
| Context Length | 128 |
| Training Steps | 3,000 |
| Final Val Loss | 0.9235 |
Architecture
Standard mini-GPT architecture:
- Token Embedding + Position Embedding
- 6 Transformer Blocks (Causal Self-Attention + MLP)
- LayerNorm + Residual Connections
- Weight Tying (input embedding = output projection)
Training
Trained on CPU for ~1 hour 17 minutes on the TinyStories dataset.
- Optimizer: AdamW (lr=3e-4, weight_decay=0.01)
- Schedule: Warmup (200 steps) + Cosine Decay
- Batch Size: 64, Block Size: 128
Usage
import torch
from train_torch import MiniGPT
# Load model
model = MiniGPT(vocab_size=72, n_embd=256, n_head=8, n_layer=6, block_size=128, dropout=0.1)
model.load_state_dict(torch.load("model.safetensors"))
model.eval()
# Generate
context = torch.zeros((1, 1), dtype=torch.long)
output = model.generate(context, max_new_tokens=200, temperature=0.7)
Sample Generation (Temperature=0.7)
He saw a bike with a big box. He was a girl named Sue. Sue liked to give it very happy. One day, she got to the little boy named Tim. Tim was very happy. He wind the best friends was friends. The tree...
License
MIT
- Type
- minigpt
- License
- mit
- Languages
- en
- Created
- Jul 27, 2026
- Updated
- Jul 27, 2026