Downloads · 30 days
3
8% of all-time downloads
ULTRON-Core-512M
HR26kk/ULTRON-Core-512M
ULTRON-Core-512M is a machine learning model from HR26kk. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
ULTRON-Core-512M is a multilingual foundation model pre-trained from scratch across 81.9 Million total tokens (65.5M domain tokens + 16.4M educational science/math tokens from cosmopedia-100k) on 2x Tesla T4 GPUs.
All-time downloads
37
Public
Parameters
562M
2.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.1 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- ultron_core
- Access
- Public
- Created
- Aug 21, 2026
- Updated
- Sep 4, 2026
- SHA
- 80a2b698
Model card
From the Hugging Face model README
ULTRON-Core-512M Foundation Model (Generation 3 - CPT Knowledge Enhanced)
ULTRON-Core-512M is a multilingual foundation model pre-trained from scratch across 81.9 Million total tokens (65.5M domain tokens + 16.4M educational science/math tokens from cosmopedia-100k) on 2x Tesla T4 GPUs.
Specifications
- Total Parameters: 562,382,848 (~512.05M Non-Embedding)
- Architecture: Multi-Head Latent Attention (MLA: $d_c=512, d_R=64$) + QK-LayerNorm + SwiGLU Gating + RoPE ($\theta=500k$)
- Hidden Size ($d_{model}$): 1024 (36 Layers)
- Vocabulary Size ($V$): 24,576 (Pure-Python Byte-Level BPE, NFC Standard)
- Total Tokens Trained: 81,920,000 tokens
- Final CPT Loss: 3.1075 (Perplexity: 22.36)
- Type
- ultron_core
- License
- apache-2.0
- Languages
- en, hi, multilingual
- Created
- Aug 21, 2026
- Updated
- Sep 4, 2026