Downloads · 30 days
24
24% of all-time downloads
gpt2-124m-ablation
bitlabsdb/gpt2-124m-ablation
gpt2-124m-ablation is a text generation model from bitlabsdb. Use it when you need the model to write or continue text. It is set up for pytorch. The card lists the license as mit.
This is a 124M parameter Causal Language Model (GPT-2 Small architecture) trained entirely from scratch using PyTorch.
All-time downloads
102
Public
Parameters
163M
1.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin653 MB · 50%
At a glance
- Task
- Text Generation
- Library
- pytorch
- License
- mit
- Model type
- gpt2
- Access
- Public
- Created
- Nov 25, 2025
- Updated
- Nov 26, 2025
- SHA
- 89d35bbe
Try a prompt
Datasets
Model card
From the Hugging Face model README
GPT-2 (124M) - Research Ablation Baseline
Model Summary
This is a 124M parameter Causal Language Model (GPT-2 Small architecture) trained entirely from scratch using PyTorch.
It was created as a baseline for a research ablation study to investigate training dynamics, achieving a validation loss of 4.485.
Model Details
- Architecture: Custom GPT-2 Small (Decoder-only Transformer)
- Parameters: 124M
- Context Window: 256 tokens
- Dimensions: Embedding: 768, Heads: 12, Layers: 12
- Training Steps: ~28,500
- Validation Loss: 4.485
How to Use
⚠️ Important: Because this model was trained using a custom PyTorch class (not the standard Hugging Face GPT2LMHeadModel), you must define the model architecture in your code before loading the weights.
- Task
- Text Generation
- Library
- pytorch
- Type
- gpt2
- License
- mit
- Languages
- en
- Created
- Nov 25, 2025
- Updated
- Nov 26, 2025