Downloads · 30 days
7
12% of all-time downloads
GPTNeoX-160M-MiniPile-cmu-11667-hw5
caizefeng18/GPTNeoX-160M-MiniPile-cmu-11667-hw5
GPTNeoX-160M-MiniPile-cmu-11667-hw5 is a machine learning model from caizefeng18. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
All-time downloads
57
Public
Parameters
162M
325 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors325 MB · 99%
At a glance
- License
- mit
- Model type
- gpt_neox
- Access
- Public
- Created
- Nov 15, 2024
- Updated
- Nov 15, 2024
- SHA
- 8cd525e9
Base models
Model card
From the Hugging Face model README
Training Hyperparameters
{
"model_to_train": "models/GPTNeoX-160M-WikiText-512-flash_attention_2-2048-gradient_checkpointing-FT",
"output_dir": "...",
"run_name": "...",
"report_to": "wandb",
"eval_strategy": "epoch",
"logging_steps": 1,
"num_train_epochs": 3,
"save_strategy": "no",
"bf16": true,
"seq_len": 2048,
"attention_type": "flash_attention_2",
"gradient_checkpointing": true,
"dataset": "minipile",
"deepspeed": null,
"per_device_train_batch_size": 24,
"learning_rate": 0.0016,
"weight_decay": 0.0,
"warmup_ratio": 0.1,
"lr_scheduler_type": "cosine_with_min_lr",
"lr_scheduler_kwargs": {
"min_lr": 1e-6
}
}
- Type
- gpt_neox
- License
- mit
- Created
- Nov 15, 2024
- Updated
- Nov 15, 2024