Downloads · 30 days
14
17% of all-time downloads
Cascade0.1-Preview1-0505-Step55K
ARMZyany/Cascade0.1-Preview1-0505-Step55K
Cascade0.1-Preview1-0505-Step55K is a text generation model from ARMZyany. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
All-time downloads
84
Public
Parameters
170M
679 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors679 MB · 100%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- May 5, 2026
- Updated
- May 5, 2026
- SHA
- a61acbfc
Try a prompt
Datasets
Model card
From the Hugging Face model README
Cascade0.1 Preview1-55K
An early release of Cascade0.1, thus it does NOT represent the final state of the model.
This is the third iteration of the Cascade0.1 Series- First it was in February which went horribly, and now in May. The third iteration from may utilizes the same custom pipeline used in the making of Cascade0, but heavily updated. As we speak, the fourth iteration is in training and its TBD if that will remain the final training script.
What changed vs Cascade0
Well the param config in C0.1 now prioritizes actual intelligence (and speed), both reaching 170M param size
| Parameter | Cascade 0.1 | Cascade 0 |
|---|---|---|
| Vocab Size | 32,000 | 56,000 |
| Hidden Size | 896 | 768 |
| Intermediate Size | 2,384 | 2,248 |
| Number of Layers | 16 | 16 |
| Attention Heads | 14 | 14 |
| KV Heads | 7 | - |
| Max Position Embeddings | 1,512 | 1,512 |
| Tie Word Embeddings | True | False |
| Use Cache | False | False |
It fixes the biggest bug of the base Cascade0 which was the presence of instruct datasets in raw pretraining. Cascade0.1 utilizes only a pure text database, which is a far better base for future SFT. Another fix is the lowercase only bug from the tokenizer
As shown in the benchmarks below, this preview is very close to its 'father' (Cascade0-Base), despite having 355K less steps into training. (C0-Base had 400K steps in in 2 weeks. C0.1 took 4 hours and 55K steps) ( and its all pure raw text :ppppp )
because this is a preview, i wont release any GGUFs for now
- Task
- Text Generation
- Type
- llama
- License
- apache-2.0
- Languages
- en
- Created
- May 5, 2026
- Updated
- May 5, 2026