Downloads · 30 days
31
16% of all-time downloads
Doge-40M-checkpoint
SmallDoge/Doge-40M-checkpoint
Doge-40M-checkpoint is a text generation model from SmallDoge. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Doge uses wsdscheduler as the training scheduler, which divides the learning rate into three stages: warmup, stable, and decay. It allows us to continue training on any new dataset from any checkpoint in the stable st…
All-time downloads
188
Public
Parameters
40.7M
424 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors81.4 MB · 83%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- doge
- Access
- Public
- Created
- Aug 12, 2025
- Updated
- Aug 13, 2025
- SHA
- 43eec4bf
Try a prompt
Datasets
Model card
From the Hugging Face model README
Doge 40M checkpoint
Doge uses wsd_scheduler as the training scheduler, which divides the learning rate into three stages: warmup, stable, and decay. It allows us to continue training on any new dataset from any checkpoint in the stable stage without spikes in training.
Here are the initial learning rates required to continue training at each checkpoint:
- Doge-40M: 8e-3
- Doge-40M-MoE: 8e-3
| Model | Learning Rate | Schedule | Warmup Steps | Stable Steps |
|---|---|---|---|---|
| Doge-40M | 8e-3 | wsd_scheduler | 2000 | 4000 |
| Doge-40M-MoE | 8e-3 | wsd_scheduler | 2000 | 4000 |
- Task
- Text Generation
- Library
- transformers
- Type
- doge
- License
- apache-2.0
- Languages
- en, zh
- Created
- Aug 12, 2025
- Updated
- Aug 13, 2025