Downloads · 30 days
0
Stryda-124M-TinyStories
AndreeFrby/Stryda-124M-TinyStories
Stryda-124M-TinyStories is a machine learning model from AndreeFrby. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
I trained this Language Model from scratch to understand how the architecture works internally. It is a 124 million parameter model based on the GPT-2 architecture, trained on the TinyStories dataset.
Access
Public
Updated Dec 4, 2025
Repo size
1.5 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.pt1.5 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Dec 4, 2025
- Updated
- Dec 4, 2025
- SHA
- 38c9cd2f
Model card
From the Hugging Face model README
I trained this Language Model from scratch to understand how the architecture works internally. It is a 124 million parameter model based on the GPT-2 architecture, trained on the TinyStories dataset.
I used Andrej Karpathy's nanoGPT repository as a base and trained it on a single RTX 5060 Ti using Vast.ai.
Model Details Architecture: GPT-2 style (Decoder-only Transformer)
Parameters: 124M
Layers: 12
Heads: 12
Embedding Dimension: 768
Dataset: TinyStories
Context Window: 1024 tokens
Usage This model is designed to generate short, coherent English stories with simple vocabulary. It is not a chat assistant, but a completion model.
Reflection I am Andre, a prompt engineer and AI Red Teamer. I wanted to see if I could build and train a functional LLM completely from zero. It took some trial and error, but it works.
If I was able to do this at my 14yo, you can do it too. It is never too late to learn how these things work behind the scenes.
- License
- apache-2.0
- Languages
- en
- Created
- Dec 4, 2025
- Updated
- Dec 4, 2025