Downloads · 30 days
0
Accurate6DigitSubtraction
PhilipQuirke/Accurate6DigitSubtraction
Accurate6DigitSubtraction is a machine learning model from PhilipQuirke. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Contains files for a Transformer model that answers 6-digit subtraction questions (e.g. 123450-345670=-0123230) with very low loss (1e-8).
Access
Public
Updated Apr 29, 2024
Repo size
1 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.pth290 MB · 98%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Dec 27, 2023
- Updated
- Apr 29, 2024
- SHA
- 49301859
Model card
From the Hugging Face model README
Contains files for a Transformer model that answers 6-digit subtraction questions (e.g. 123450-345670=-0123230) with very low loss (1e-8).
This subtraction model has 3 layers, 4 attention heads, d-model = 510, d-head = 170. The subtraction model was initialised with a very-low-loss Addition model (2 layers, 3 attention heads, 9e-9 loss), before being trained for 45K epochs.
The CoLab used to train the model is here: https://github.com/apartresearch/Verified_addition/blob/main/assets/Accurate_Math_Train.ipynb
The CoLab used to analyse the model is here: https://github.com/apartresearch/Verified_addition/blob/main/assets/Accurate_Math_Analyse.ipynb
- License
- apache-2.0
- Created
- Dec 27, 2023
- Updated
- Apr 29, 2024