Downloads · 30 days
0
vaayu-94M
vprojectx/vaayu-94M
vaayu-94M is a machine learning model from vprojectx. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Vayu-ML is a compact, custom decoder-only Transformer pre-trained from scratch on 500M tokens and aligned via Supervised Fine-Tuning (SFT) on ML/DL theoretical concepts.
Access
Public
Updated Sep 18, 2026
Parameters
135M
982 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors541 MB · 55%
At a glance
- License
- mit
- Access
- Public
- Created
- Sep 18, 2026
- Updated
- Sep 18, 2026
- SHA
- 16c42890
Model card
From the Hugging Face model README
Vayu-ML (94M Parameters)
Vayu-ML is a compact, custom decoder-only Transformer pre-trained from scratch on 500M tokens and aligned via Supervised Fine-Tuning (SFT) on ML/DL theoretical concepts.
Model Architecture
- Parameters: ~94 Million
- Layers: 12
- Hidden Dimension: 768
- Intermediate Dimension (SwiGLU): 2048
- Attention Heads: 12
- Normalization: RMSNorm
- Vocabulary Size: 32,768 (Custom BPE Tokenizer)
- Context Length: 512 tokens
Training Details
- Pretraining: ~500M tokens in native
bfloat16using AdamW. - Alignment: SFT on curated ML/DL conceptual Q&A with causal response masking.
- License
- mit
- Languages
- en
- Created
- Sep 18, 2026
- Updated
- Sep 18, 2026