Downloads · 30 days
9
10% of all-time downloads
GPTNeoX-160M-MiniPile-2048-flash-gradient
ddiaz312/GPTNeoX-160M-MiniPile-2048-flash-gradient
GPTNeoX-160M-MiniPile-2048-flash-gradient is a machine learning model from ddiaz312. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
GPTNeoX-160M MiniPile Training Configuration: { "modeltotrain": "models/GPTNeoX-160M-WikiText-2048-flash-gradient", "outputdir": "models/GPTNeoX-160M-MiniPile-2048-flash-gradient", "runname": "GPTNeoX-160M-MiniPile-20…
All-time downloads
94
Public
Parameters
162M
325 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors325 MB · 99%
At a glance
- Model type
- gpt_neox
- Access
- Public
- Created
- Nov 14, 2024
- Updated
- Nov 14, 2024
- SHA
- 043d87d4
Model card
From the Hugging Face model README
GPTNeoX-160M MiniPile Training Configuration:
{
"model_to_train": "models/GPTNeoX-160M-WikiText-2048-flash-gradient",
"output_dir": "models/GPTNeoX-160M-MiniPile-2048-flash-gradient",
"run_name": "GPTNeoX-160M-MiniPile-2048-flash-gradient",
"report_to": "wandb",
"eval_strategy": "epoch",
"logging_steps": 1,
"num_train_epochs": 1,
"save_strategy": "no",
"bf16": true,
"seq_len": 2048,
"attention_type": "flash_attention_2",
"gradient_checkpointing": true,
"dataset": "minipile",
"deepspeed": null,
"per_device_train_batch_size": 8,
"learning_rate": 0.001,
"weight_decay": 0.05,
"warmup_ratio": 0.03,
"lr_scheduler_type": "linear",
"hyperparameter_tuning": true
}
- Type
- gpt_neox
- Created
- Nov 14, 2024
- Updated
- Nov 14, 2024