Downloads · 30 days
0
tiny-gpt-lab-v0.2-broadened-lightfineweb
cnoles/tiny-gpt-lab-v0.2-broadened-lightfineweb
tiny-gpt-lab-v0.2-broadened-lightfineweb is a text generation model from cnoles. Use it when you need the model to write or continue text. It is set up for pytorch. The card lists the license as mit.
This is the current best broadened checkpoint from the tiny GPT home-lab project.
Access
Public
Updated Mar 6, 2026
Repo size
571 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt571 MB · 100%
At a glance
- Task
- Text Generation
- Library
- pytorch
- License
- mit
- Access
- Public
- Created
- Mar 6, 2026
- Updated
- Mar 6, 2026
- SHA
- dca31175
Try a prompt
Model card
From the Hugging Face model README
tiny-gpt-lab-v0.2-broadened-lightfineweb
Summary
This is the current best broadened checkpoint from the tiny GPT home-lab project.
It continues a strong TinyStories + Cosmopedia base with a lighter FineWeb-Edu broadening blend. The goal of this stage was to widen the model's base knowledge without losing too much story quality.
Source
- Source run:
runs/tinystories-cosmopedia-fineweb-light-vocab-8192-gpu - Dataset blend: TinyStories + Cosmopedia + FineWeb-Edu (
6:2:1) - Tokenizer: BPE, vocab
8192
Model
n_layers=12n_heads=8n_embd=512block_size=512
Training Result
- Best validation loss:
1.3451at step95000 - Final validation loss:
1.3644
Known Limitations
- Output quality is still inconsistent on longer or more expository prompts.
- The model can drift into templated educational language.
- This is still an experimental local model, not a production assistant.
- Task
- Text Generation
- Library
- pytorch
- License
- mit
- Languages
- en
- Created
- Mar 6, 2026
- Updated
- Mar 6, 2026