Downloads · 30 days
0
gpt2_124M_1BTokens
mtmx/gpt2_124M_1BTokens
gpt2_124M_1BTokens is a machine learning model from mtmx. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
This is a GPT-2 model trained from scratch on [your dataset].
Access
Public
Updated Nov 27, 2025
Repo size
24.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt24.7 GB · 100%
At a glance
- License
- mit
- Access
- Public
- Created
- Nov 7, 2025
- Updated
- Nov 27, 2025
- SHA
- f0dc9645
Model card
From the Hugging Face model README
GPT-2 Model - Trained from Scratch
This is a GPT-2 model trained from scratch on [your dataset].
Model Details
- Architecture: GPT-2
- Parameters: 124M
- Training Steps: 19073
- Validation Loss: 4.21
- HellaSwag Accuracy: 25.58%
Training Configuration
- Batch Size: 32
- Sequence Length: 1024
- Learning Rate: 6e-4
- Total Tokens: ~1B tokens
Usage
import torch
# Load checkpoint
checkpoint = torch.load('model_hf_09000.pt') #current saved checkpoint, to be updated after fully training
config = checkpoint['config']
# Initialize model
from your_model_file import GPT # Your GPT class
model = GPT(config)
model.load_state_dict(checkpoint['model'])
model.eval()
# Generate text
# ... your generation code ...
Training Details
Trained using custom GPT-2 implementation following Andrej Karpathy's approach.
- License
- mit
- Languages
- en
- Created
- Nov 7, 2025
- Updated
- Nov 27, 2025