Downloads · 30 days
4
10% of all-time downloads
HobbyLM-30M
harims95/HobbyLM-30M
HobbyLM-30M is a machine learning model from harims95. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
A 31.9M parameter dense transformer trained from scratch on 1B tokens of FineWeb.
All-time downloads
42
Public
Parameters
31.9M
128 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors128 MB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Jun 27, 2026
- Updated
- Jun 27, 2026
- SHA
- 5692ab31
Model card
From the Hugging Face model README
HobbyLM-30M
A 31.9M parameter dense transformer trained from scratch on 1B tokens of FineWeb.
Built on top of HobbyLM by rootxhacker.
Training
- Parameters: 31.9M (fully dense)
- Dataset: FineWeb (1B tokens)
- Steps: 3800
- Final val loss: 3.9077
- Architecture: 8 layers, d_model=384, 6 heads, GQA, RoPE, RMSNorm, Muon optimizer
This is a base model
No instruction tuning. Generates fluent English but drifts off topic — expected at this scale.
- License
- apache-2.0
- Languages
- en
- Created
- Jun 27, 2026
- Updated
- Jun 27, 2026