Downloads · 30 days
74
37% of all-time downloads
VegaLM1-42M-Base
CNWPlayer/VegaLM1-42M-Base
VegaLM1-42M-Base is a machine learning model from CNWPlayer. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Note: AI was used in the creation of this project. But that's why you're here, isn't it?
All-time downloads
201
Public
Parameters
42.1M
3 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.pt2 GB · 96%
At a glance
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Aug 21, 2026
- Updated
- Aug 30, 2026
- SHA
- 96a3b3b1
Datasets
Spaces
Model card
From the Hugging Face model README
VegaLM1-42M-Base
Note: AI was used in the creation of this project. But that's why you're here, isn't it?
This is the base model for VegaLM1-42M, an experimental SLM trained on various corpi... corpuses... datasets. It's not Fable 6, but it's good enough, ok?
The first 262M tokens the model saw came from a 90M selection of fineweb-edu The next 688M was from a 50/25/15/10 split of fineweb-edu, fineweb, wikipedia, and a replay respectively, totaling around 420M tokens The final 2B tokens came from a 60/25/15 split of stack-v3-train, dclm-baseline-1.0, and smollm-corpus, trained for 1 epoch because I'm lazy
Benchmarks
Final version gets 944 elo on Bananamind Base Bench 1.1.
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| arc_challenge | 1 | none | 0 | acc | ↑ | 0.1800 | ± | 0.0112 |
| none | 0 | acc_norm | ↑ | 0.2125 | ± | 0.0120 | ||
| arc_easy | 1 | none | 0 | acc | ↑ | 0.3763 | ± | 0.0099 |
| none | 0 | acc_norm | ↑ | 0.3523 | ± | 0.0098 | ||
| hellaswag | 1 | none | 0 | acc | ↑ | 0.2629 | ± | 0.0044 |
| none | 0 | acc_norm | ↑ | 0.2686 | ± | 0.0044 | ||
| piqa | 1 | none | 0 | acc | ↑ | 0.5702 | ± | 0.0116 |
| none | 0 | acc_norm | ↑ | 0.5762 | ± | 0.0115 |
Arithmark 3:
- Raw continuation accuracy 34.50%
- Length-normalized accuracy 33.90%
- Primary (acc_norm) 33.90%
Model specs
- Layers: 12
- Hidden size: 512
- Attention heads: 8
- KV heads: 4
- Context length: 2,048 tokens
- Intermediate size: 880
- Vocabulary: 32,000
- Parameters: 42,054,144
- Type
- llama
- License
- mit
- Languages
- en
- Created
- Aug 21, 2026
- Updated
- Aug 30, 2026