Downloads · 30 days
0
gpt_ua
nikiandr/gpt_ua
gpt_ua is a machine learning model from nikiandr. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Model weights for GPT-like decoder-only Transformer.
Access
Public
Updated Aug 15, 2023
Repo size
232 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt232 MB · 100%
At a glance
- Access
- Public
- Created
- Aug 15, 2023
- Updated
- Aug 15, 2023
- SHA
- 3363404f
Datasets
Model card
From the Hugging Face model README
Model weights for GPT-like decoder-only Transformer.
Configuration used for gpt_64_bs128_5000epochs_lr1.0e-03_8heads_emb512.pt:
BATCH_SIZE = 64
SEED = 42
BLOCK_SIZE = 128
EPOCHS = 5000
TRAIN_SUBSET_LENGTH = None # 10_000_000
TRAIN_PERC = 0.99
EVAL_PERIOD = 500
EVAL_ITERS = 100
EMBED_SIZE = 512
NUM_HEADS = 8
LEARNING_RATE = 1e-3
BLOCK_NUMBER = 8
Code repo: nikiandr/gpt_ua.
- Created
- Aug 15, 2023
- Updated
- Aug 15, 2023