Downloads · 30 days
0
SymbioGPT-10M
LisaMegaWatts/SymbioGPT-10M
SymbioGPT-10M is a machine learning model from LisaMegaWatts. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Multi-organelle GPT language model (11.6M params) trained on classical philosophy texts.
Access
Public
Updated Mar 3, 2026
Repo size
1.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt1.5 GB · 99%
At a glance
- License
- mit
- Access
- Public
- Created
- Feb 27, 2026
- Updated
- Mar 3, 2026
- SHA
- 215b74bc
Model card
From the Hugging Face model README
SymbioGPT-10M
Multi-organelle GPT language model (11.6M params) trained on classical philosophy texts.
Architecture
SymbioGPT extends the SymbioSLM architecture by adding CausalSelfAttention as a 4th organelle, all fused via a learned per-channel OrganelleGate with learnable temperature.
| Organelle | Function | Complexity |
|---|---|---|
| CausalDepthwiseConv1d | Local n-gram detection | O(n) |
| MonarchMatrix | Sub-quadratic global mixing via butterfly matrices | O(n√n) |
| LongConv | Dense causal convolution with exponential decay | O(n) |
| CausalSelfAttention | Multi-head attention with RoPE | O(n²) |
Plus: RMSNorm, SwiGLU FFN, SkipGate residuals, weight-tied output projection.
Model Details
| Parameter | Value |
|---|---|
| d_model | 320 |
| n_layers | 8 |
| n_heads | 5 |
| head_dim | 64 |
| context_length | 256 |
| vocab_size | 2000 (BPE) |
| n_monarch_heads | 1 |
| Total params | 11.6M |
Files
symbio_best.pt— Best checkpoint (PyTorch state_dict, torch.compile format)symbio_final.pt— Final checkpointvocab.json— BPE vocabulary (2000 tokens)merges.txt— BPE merge rules
Usage
Try it live: SymbioGPT-10M Space
Links
- Source: DavinciDreams/symbiogenesis
- Training data: Classical philosophy corpus (Aristotle, Plato, Seneca, Marcus Aurelius, etc.)
- License
- mit
- Languages
- en
- Created
- Feb 27, 2026
- Updated
- Mar 3, 2026