Downloads · 30 days
2
7% of all-time downloads
mathstral-nano-sat
vivicake666/mathstral-nano-sat
mathstral-nano-sat is a machine learning model from vivicake666. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
A tiny GPT-style causal language model (236,928 parameters) trained on SAT-level math problems. Built entirely in NumPy with no PyTorch dependency. Demonstrates the full fine-tuning pipeline: tokenization, causal atte…
All-time downloads
30
Public
Parameters
237K
952 KB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors952 KB · 99%
At a glance
- License
- apache-2.0
- Model type
- mathstral_nano
- Access
- Public
- Created
- Mar 8, 2026
- Updated
- Mar 8, 2026
- SHA
- d4196d6f
Model card
From the Hugging Face model README
mathstral-nano-sat
A tiny GPT-style causal language model (236,928 parameters) trained on SAT-level math problems. Built entirely in NumPy with no PyTorch dependency. Demonstrates the full fine-tuning pipeline: tokenization, causal attention, AdamW, and cross-entropy loss on a byte-level vocabulary.
Model Details
| Property | Value |
|---|---|
| Architecture | 4-layer causal transformer |
| Attention heads | 8 |
| Hidden dim | 64 |
| FFN dim | 256 |
| Vocabulary | 256 (byte-level, UTF-8) |
| Max seq length | 64 |
| Total parameters | 236,928 |
| Framework | Pure NumPy + SciPy |
Training
| Property | Value |
|---|---|
| Dataset | 20 SAT math Q&A examples |
| Epochs | 3 |
| Steps | 300 |
| Batch size | 8 |
| Learning rate | 0.0003 (AdamW) |
| Baseline loss | 5.5158 |
| Final loss | 2.224 |
| Loss reduction | 59.7% |
Install
pip install safetensors scipy numpy
Usage
from modeling_mathstral_nano import MathstralNano
model = MathstralNano.from_pretrained(".")
print(model)
# MathstralNano(4L 8H 64d params=236,928)
# Raw generation
response = model.generate("Problem: If 2x + 5 = 13, find x. Solution:")
print(response)
# Convenience wrapper (formats the prompt automatically)
response = model.solve("If 3x + 7 = 22, find x.")
print(response)
Files
| File | Description |
|---|---|
model.safetensors | Weights in safetensors format |
config.json | Architecture config |
modeling_mathstral_nano.py | Pure-NumPy model class with inference |
training_metadata.json | Full training run metadata and loss curve |
- Type
- mathstral_nano
- License
- apache-2.0
- Languages
- en
- Created
- Mar 8, 2026
- Updated
- Mar 8, 2026