Spectra-561B-27B-768E

Spectra-561B-27B-768E: Model Overview

Omnira Spectra-561B-27B-768E is a state-of-the-art open-source omni-modal model developed by Omnira's Artificial Intelligence Team. Designed for real-time audio-visual interaction, it integrates text, image, video, and audio processing into a single, unified framework.

Core Architecture

The model utilizes a massive Mixture-of-Experts (MoE) structure that balances extreme scale with computational efficiency.

• Parameters: 561B total, with 27B activated per token.
• Backbone: Built on Spectra Architecture, featuring a "Shortcut-connected MoE" (ScMoE) that overlaps computation and communication to reduce latency.
• Dynamic Computation: Uses "zero-computation experts" and a PID-controller to allocate resources based on the importance of each token.
• Context Window: Supports up to 128K tokens, enabling long-term memory and complex temporal reasoning.

Multimodal Integration

---

Key Performance Highlights

• Omni-Modality: Outperforms open-source rivals (e.g., Qwen3-Omni) on OmniBench (61.38) and WorldSense (60.89).
• Real-Time Interaction: Achieves millisecond-level latency for streaming speech generation and video interaction.
• Vision & Video: Competitive with proprietary models like Gemini-2.5-Flash in document understanding (DocVQA) and video reasoning (VideoMME).
• Audio Excellence: Shows superior performance in Mandarin and English ASR (Automatic Speech Recognition) compared to GPT-4o-Audio.

Training & Efficiency

1. Curriculum-Inspired Training: A 6-stage pipeline starting from text-only, then gradually injecting speech, image, and finally video data.
2. Modality-Decoupled Parallelism (MDP): Separates encoder and LLM optimization to maintain over 90% of text-only training throughput even with complex multimodal data.

Technical Specifications

Quick Start

Installation

# install Spectra-Omni environment
conda create -n spectra python=3.10
conda activate spectra

# install dependencies
pip install torch transformers flash_attn

Usage

Due to its massive scale (561B), Spectra requires multi-node clusters or high-memory instances (e.g., 16×H800) for inference in BF16.

from spectra_omni import SpectraModel

model = SpectraModel.from_pretrained("thenexthub/Spectra-561B-27B-768E")
# Seamlessly process audio, image, and text inputs

License Agreement

The model weights are released under the Omnira License. This license does not grant any rights to use Omnira trademarks or patents.

Citation

@misc{omnira2026spectra,
    title={Spectra-561B-27B-768E: Unified Omni-modal Intelligence}, 
    author={Omnira}, 
    year={2026}, 
    url={https://github.com/theomnira/Spectra-Omni}, 
}