sage-lumen-3m

📐 SAGE-Lumen-3M: Sovereign State-Space Transition Engine

SAGE-Lumen-3M is an ultra-compact, parameter-disciplined 3 million parameter language model optimized for offline edge execution on homelab nodes (e.g., AMD Ryzen APUs).

Unlike general-purpose chatbots, SAGE-Lumen-3M is built strictly as a Symbolic Physical Reasoning and State-Space Transition Simulator. It models trajectories across three primary domains:

1. Discrete-Time Quantum Walks (DTQW): Simulating 1-D position states under Hadamard-coin operations.
2. Biophysical ODE Trajectories: Homeostatic insulin-glucose couplings and islet cell neovascularization.
3. Non-Planar G-code Toolpaths: Predicting Z-axis warp corrections dynamically against mechanical thermals.

🧠 Architectural Specifications

• Parameters: 2,754,816 active parameters (Tied Embeddings)
• Vocabulary Size: 2,048 (Capped BPE)
• Decoder Layers: 4 layers
• Attention Heads: 4 query heads, 1 Key-Value head (Multi-Query Attention)
• Position Embeddings: Rotary Positional Embeddings (RoPE)
• Feed-Forward Blocks: SwiGLU MLP (SiLU-gated, intermediate dimension 512)
• Normalization: RMSNorm (Root Mean Square Layer Normalization)
• Training Optimization: Matrix-preconditioned momentum (Muon Optimizer on 2D projections, AdamW on 1D variables).

🚀 Quickstart Usage (PyTorch)

import torch
# Place the model on your local target device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

For full details on SAGE (Sovereign Agentic Governance & Epistemic Protocol) or Logos OS, check our Github repository.