Downloads · 30 days
70
18% of all-time downloads
Vega-1.6-128M-Base
CNWPlayer/Vega-1.6-128M-Base
Vega-1.6-128M-Base is a machine learning model from CNWPlayer. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Note: AI was used in the creation of this project. But that's why you're here, isn't it?
All-time downloads
380
Public
Parameters
128M
1.3 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors257 MB · 99%
At a glance
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Aug 29, 2026
- Updated
- Sep 5, 2026
- SHA
- 4cba7243
Model card
From the Hugging Face model README
Vega-1.6-128M-Base
Note: AI was used in the creation of this project. But that's why you're here, isn't it?
This is the base model for Vega-1.6-128M, an experimental SLM trained on various datasets. (Ha! I've learned from last time!)
First corpus was 3B toks of 33.3/33.3/33.3 fineweb, fineweb-edu, and cosmopedia. Second corpus was 5B tokens taken from a 35/30/20/15 split of NPset-2-Python-Edu, open-web-math, cosmopedia, and fineweb-edu.
Benchmarks
1019 elo on Bananamind Base Bench 1.1, putting it in 17th place as of writing, ahead of GPT-X-125M and behind Supra2-100M.
- Hellaswag: 29.62%
- ARC-E: 39.64%
- ARC-C: 23.89%
- PIQA: 59.08%
- Arithmark-3: 35.60%
Model specs
- Layers: 16
- Hidden size: 768
- Attention heads: 12
- KV heads: 6
- Context length: Trained on 2,048 tokens
- Intermediate size: 2048
- Vocabulary: 32,000
- Parameters: 128,410,368
- Type
- llama
- License
- mit
- Languages
- en
- Created
- Aug 29, 2026
- Updated
- Sep 5, 2026