Downloads · 30 days
20
28% of all-time downloads
NanoGPT2-FP128
FlameF0X/NanoGPT2-FP128
NanoGPT2-FP128 is a machine learning model from FlameF0X. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
A nano GPT-2 style causal language model trained on TinyStories with double-double (~FP128) arithmetic in the forward pass.
All-time downloads
72
Public
Parameters
41K
330 KB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors330 KB · 98%
At a glance
- License
- mit
- Model type
- nano_gpt2_fp128
- Access
- Public
- Created
- Mar 15, 2026
- Updated
- Mar 15, 2026
- SHA
- 03da680d
Datasets
Model card
From the Hugging Face model README
nano-gpt2-fp128
A nano GPT-2 style causal language model trained on TinyStories with double-double (~FP128) arithmetic in the forward pass.
Architecture
| Hyper-parameter | Value |
|---|---|
| Embedding dim | 32 |
| Attention heads | 2 |
| Transformer layers | 2 |
| Context window | 64 |
| Vocabulary | 82 (char-level, TinyStories) |
| Parameters | 32,768 |
Precision
| Stage | Precision |
|---|---|
| Weight storage | float64 |
| Forward matmuls | ~106-bit (double-double / Veltkamp split) |
| Backward pass | float64 |
| Equivalent to | IEEE binary128 (113-bit) within 7 bits |
- Type
- nano_gpt2_fp128
- License
- mit
- Languages
- en
- Created
- Mar 15, 2026
- Updated
- Mar 15, 2026