Spektron

Spektron

Foundation model for vibrational spectroscopy using D-LinOSS (Damped Linear Oscillatory State Space) models.

Model Description

Spektron is a state space model designed specifically for vibrational spectroscopy (IR, Raman). It combines:

• D-LinOSS Architecture: Second-order oscillatory SSM with damped sinusoid basis, naturally suited for vibrational spectra
• Masked Pretraining: Self-supervised learning on 130K quantum chemistry spectra (QM9S dataset)
• Variational Information Bottleneck (VIB): Disentangles chemical information from instrument factors
• Cross-Spectral Prediction: Novel capability to predict Raman from IR spectra and vice versa

Key Features

• 🔬 Spectroscopy-Native: D-LinOSS oscillatory dynamics match the physics of molecular vibrations
• 📊 Data-Efficient: Achieves strong performance with <100 labeled samples via masked pretraining
• 🔄 Bidirectional Scanning: Forward + backward D-LinOSS layers for full spectral context
• ⚡ Efficient: 6.6M parameters, O(n) complexity (vs O(n²) for Transformers)
• 🎯 Interpretable: Transfer function H(z) analysis reveals learned spectral filters

Model Architecture

Spectrum (B, 2048)
  → RawSpectralEmbedding    # Conv1d + wavenumber PE + [CLS] + [DOMAIN]
  → D-LinOSS Backbone       # 4 bidirectional layers, 256 hidden dim
  → MixtureOfExperts        # 4 experts, top-2 gating
  → TransformerEncoder      # 2 blocks
  → VIBHead                 # z_chem (128d) + z_inst (64d)
  → Task Heads              # Reconstruction | CrossSpectral | Regression

Parameters: 6.6M total (4.7M backbone + 1.9M heads)

Training Data

• Dataset: QM9S - 130K organic molecules with DFT-computed IR/Raman spectra
• Spectral Range: 500-4000 cm⁻¹, 2048 points
• Method: B3LYP/def2-TZVP (Gaussian 16)
• Molecules: Non-centrosymmetric (99.93%), C/H/N/O/F composition

Performance

Intended Use

Primary Applications:

• Spectral preprocessing and denoising
• Cross-spectral prediction (IR ↔ Raman)
• Calibration transfer (instrument adaptation)
• Few-shot learning for new spectroscopic tasks
• Feature extraction for downstream chemometrics

Research Areas:

• Vibrational spectroscopy
• Chemical analysis
• Materials characterization
• Quality control
• Pharmaceutical analysis

Limitations

• Trained only on gas-phase DFT spectra (single molecule, no intermolecular effects)
• May not generalize to condensed-phase spectra (liquids, solids) without fine-tuning
• Limited to organic molecules from QM9 distribution (C/H/N/O/F, ≤9 heavy atoms)
• Cross-spectral prediction works best for non-centrosymmetric molecules

How to Use

import torch
from spektron import Spektron, SpektronConfig

# Load pretrained model
config = SpektronConfig.from_pretrained("ktubhyam/Spektron")
model = Spektron(config)
model.load_state_dict(torch.load("checkpoint.pt"))
model.eval()

# Predict Raman from IR spectrum
with torch.no_grad():
    ir_spectrum = torch.randn(1, 2048)  # Your IR spectrum
    raman_pred = model.predict_cross_spectral(
        ir_spectrum,
        source_domain="ir",
        target_domain="raman"
    )

Training Details

Pretraining

• Objective: Masked spectral reconstruction + optimal transport loss
• Mask Ratio: Progressive 15% → 40% over first 40% of training
• Batch Size: 64 (effective, with gradient accumulation)
• Optimizer: AdamW (lr=3e-4, weight decay=0.05 on weight matrices only)
• Schedule: Cosine annealing with 5% warmup, min LR = 0.01× max
• Epochs: 50K steps (~23 GPU-hours on 2× RTX 5060 Ti)
• Precision: BF16 AMP (D-LinOSS forced to FP32 for stability)

Augmentation

• Gaussian noise (σ=0.02, p=0.5)
• Baseline drift (polynomial order 3, p=0.5)
• Intensity scaling (0.8-1.2×, p=0.5)
• Wavelength shift (±5 cm⁻¹, p=0.5)

VIB Annealing

• β: 0.1 → 1e-3 (cosine decay over first 60% of training)
• Promotes diverse representations early, tight bottleneck late

Citation

If you use Spektron in your research, please cite:

@software{karthikeyan2026spektron,
  author = {Karthikeyan, Tubhyam},
  title = {Spektron: Foundation Model for Vibrational Spectroscopy},
  year = {2026},
  publisher = {Zenodo},
  version = {v0.1.0},
  doi = {10.5281/zenodo.18802440},
  url = {https://github.com/ktubhyam/Spektron}
}

Links

• GitHub: https://github.com/ktubhyam/Spektron
• Project Page: https://tubhyam.dev/projects/spektron
• Blog Post: https://tubhyam.dev/blog/spectral-inverse-problem
• Zenodo DOI: https://doi.org/10.5281/zenodo.18802440
• Paper: Preprint in preparation (target: Digital Discovery)

License

MIT License - see LICENSE

Author

Tubhyam Karthikeyan

• Institute of Chemical Technology, Mumbai
• ORCID: 0009-0001-4424-7297
• Website: https://tubhyam.dev
• Email: [email protected]