Downloads · 30 days
12
26% of all-time downloads
sophia_520m_1
OptimizerStudy/sophia_520m_1
sophia_520m_1 is a machine learning model from OptimizerStudy. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
- Source: https://arxiv.org/abs/2509.02046 - Optimizer: sophia - Model size: 520m - Data size: 10B
All-time downloads
46
Public
Parameters
800M
3.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.2 GB · 99%
At a glance
- Model type
- llama
- Access
- Public
- Created
- Oct 24, 2025
- Updated
- Oct 24, 2025
- SHA
- 8e1016f1
Model card
From the Hugging Face model README
Model Card
- Source: https://arxiv.org/abs/2509.02046
- Optimizer:
sophia - Model size:
520m - Data size:
10B
Best configuration
| Hyperparameter | Value |
|---|---|
| beta1 | 0.95 |
| beta2 | 0.9 |
| epsilon | 1e-07 |
| gamma | 0.0125 |
| learning_rate | 0.002 |
| max_grad_norm | 1 |
| min_lr_ratio | 0 |
| train_batch_size | 128 |
| warmup | 4000 |
| weight_decay | 0.3 |
- Type
- llama
- Created
- Oct 24, 2025
- Updated
- Oct 24, 2025