Downloads · 30 days
0
tiny-stories-gpt
Here2Disrupt/tiny-stories-gpt
tiny-stories-gpt is a machine learning model from Here2Disrupt. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
This is a small GPT-like model trained from scratch on the TinyStories dataset. It was implemented using a NanoGPT-style training loop in PyTorch.
Access
Public
Updated Aug 25, 2025
Repo size
120 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt120 MB · 100%
At a glance
- License
- mit
- Access
- Public
- Created
- Aug 25, 2025
- Updated
- Aug 25, 2025
- SHA
- aab9c94e
Model card
From the Hugging Face model README
TinyStories-GPT
This is a small GPT-like model trained from scratch on the TinyStories dataset.
It was implemented using a NanoGPT-style training loop in PyTorch.
Model Details
- Architecture: 6 layers, 6 heads, 384 hidden size
- Context length: 128 tokens
- Vocab size: 50257 (GPT-2 tokenizer)
- Dataset: TinyStories
- Training: ~20k steps, AdamW, cosine LR decay
Example Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("Here2Disrupt/tiny-stories-gpt")
model = AutoModelForCausalLM.from_pretrained("Here2Disrupt/tiny-stories-gpt")
prompt = "Once upon a time"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0]))
- License
- mit
- Created
- Aug 25, 2025
- Updated
- Aug 25, 2025