Downloads · 30 days
8
36% of all-time downloads
sophia_130m_8
OptimizerStudy/sophia_130m_8
sophia_130m_8 is a machine learning model from OptimizerStudy. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
- Source: https://arxiv.org/abs/2509.02046 - Optimizer: sophia - Model size: 130m - Data size: 21B
All-time downloads
22
Public
Parameters
266M
1.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.1 GB · 98%
At a glance
- Model type
- llama
- Access
- Public
- Created
- Oct 24, 2025
- Updated
- Oct 24, 2025
- SHA
- d0f30a1e
Model card
From the Hugging Face model README
Model Card
- Source: https://arxiv.org/abs/2509.02046
- Optimizer:
sophia - Model size:
130m - Data size:
21B
Best configuration
| Hyperparameter | Value |
|---|---|
| beta1 | 0.95 |
| beta2 | 0.95 |
| epsilon | 1e-07 |
| gamma | 0.0125 |
| learning_rate | 0.002 |
| max_grad_norm | 1 |
| min_lr_ratio | 0 |
| train_batch_size | 128 |
| warmup | 4000 |
| weight_decay | 0.2 |
- Type
- llama
- Created
- Oct 24, 2025
- Updated
- Oct 24, 2025