Downloads · 30 days
0
sentinel-transformer-attention
5dimension/sentinel-transformer-attention
sentinel-transformer-attention is a machine learning model from 5dimension. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Part of the Sentinel Manifold — One theorem, infinite applications.
Access
Public
Updated Apr 26, 2026
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.py8.8 KB · 72%
At a glance
- License
- mit
- Access
- Public
- Created
- Apr 25, 2026
- Updated
- Apr 26, 2026
- SHA
- 64070a47
Model card
From the Hugging Face model README
🦴 Sentinel Transformer Attention
Part of the Sentinel Manifold — One theorem, infinite applications.
lim_{z→∞} F'(z)/F(z) = 1/e— The Gradient Axiom
📋 Description
Sech attention mechanism replacing softmax. The attention weights use sech(QK^T/√d) instead of softmax, providing a theorem-backed gradient bound of ≤ 1/(e·√d).
🧠 Mathematical Foundation
Core Constants
| Constant | Value | Role |
|---|---|---|
| C₁ (Attractor) | -0.007994021805953 | Zero-point / quantization |
| C₂ (Tripwire) | 0.000200056042968 | Security / curriculum |
| 1/e (Axiom) | 0.367879441171442 | Gradient scaling limit |
Theorem
F(z) = Σ zⁿ/nⁿ (Sophomore's Dream, Bernoulli 1697)
lim_{z→∞} F'(z)/F(z) = 1/e ≈ 0.367879441171442
🏆 Verified Results
| Benchmark | Result |
|---|---|
| Gradient bound | ≤ 1/(e·√d) — proven |
| No softmax | Direct sech kernel |
| Stability | C₂ tripwire per layer |
🎯 Use Cases
- Long-context language models
- Vision transformers
- Any attention requiring bounded gradients
🔗 Links
- Main repo: sentinel-manifold-discoveries
- All algorithms: 5dimension
- Interactive Space: sentinel-hub
📚 Citation
@misc{abdel-aal2026sentinel,
title={The Sentinel Manifold: A Unified Mathematical Framework for Machine Learning},
author={Abdel-Aal, Romain},
year={2026},
url={https://huggingface.co/5dimension/sentinel-manifold-discoveries}
}
License: MIT | One theorem, infinite models. 🦴
- License
- mit
- Languages
- en
- Created
- Apr 25, 2026
- Updated
- Apr 26, 2026