Downloads · 30 days
21.8K
2% of all-time downloads
TinyStories-33M
roneneldan/TinyStories-33M
TinyStories-33M is a text generation model from roneneldan. Use it when you need the model to write or continue text. It is set up for transformers.
Model trained on the TinyStories Dataset, see https://arxiv.org/abs/2305.07759
All-time downloads
953K
Public
Repo size
582 MB
Likes
114
Public
Repo makeup
Click a slice to open those files.
.bin291 MB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- gpt_neo
- Access
- Public
- Created
- May 12, 2023
- Updated
- Aug 12, 2024
- SHA
- 2ad0a164
Try a prompt
Datasets
Model card
From the Hugging Face model README
Model trained on the TinyStories Dataset, see https://arxiv.org/abs/2305.07759
Based on GPT-Neo architecture.
License: mit
hyperparams used to train this model:
lr = 5e-4,
lr_schedule = constant,
wd=0.1,
adam_beta1=0.9, adam_beta2 = 0.95,
context_length=512,
batch_size=80,
gradient_accumulation_steps=16
------ EXAMPLE USAGE ---
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig
model = AutoModelForCausalLM.from_pretrained('roneneldan/TinyStories-33M')
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neo-125M")
prompt = "Once upon a time there was"
input_ids = tokenizer.encode(prompt, return_tensors="pt")
# Generate completion
output = model.generate(input_ids, max_length = 1000, num_beams=1)
# Decode the completion
output_text = tokenizer.decode(output[0], skip_special_tokens=True)
# Print the generated text
print(output_text)
- Task
- Text Generation
- Library
- transformers
- Type
- gpt_neo
- Created
- May 12, 2023
- Updated
- Aug 12, 2024