Downloads · 30 days
30
41% of all-time downloads
poormans-nanogpt-tiny
gra-dient/poormans-nanogpt-tiny
poormans-nanogpt-tiny is a text generation model from gra-dient. Use it when you need the model to write or continue text. The card lists the license as mit.
A tiny 4-layer GPT model trained on TinyStories.
All-time downloads
74
Public
Parameters
16.3M
65.2 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors65.2 MB · 100%
At a glance
- Task
- Text Generation
- License
- mit
- Model type
- gpt2
- Access
- Public
- Created
- Jan 24, 2025
- Updated
- Jan 24, 2025
- SHA
- add7ce29
Try a prompt
Model card
From the Hugging Face model README
PoorMansNanoGPT-TinyStories-16M
A tiny 4-layer GPT model trained on TinyStories.
Training Details
- Dataset: TinyStories (2.1M stories)
- Hardware: 1x A100 (< 1 hour)
- Loss: 1.863 (train), 1.868 (val)
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("gra-dient/poormans-nanogpt-tiny")
tokenizer = AutoTokenizer.from_pretrained("gra-dient/poormans-nanogpt-tiny")
Acknowledgments
This project builds upon and, in some ways, is a humble tribute to the work of Andrej Karpathy nanoGPT.
As we stand on the shoulders of giants, I hope this model series contributes something and encourages others to join in the fun.
- Task
- Text Generation
- Type
- gpt2
- License
- mit
- Created
- Jan 24, 2025
- Updated
- Jan 24, 2025