thor-1-lora

THOR-1: Unified MoE Brain

THOR-1 is a LoRA adapter for Qwen3.5-35B-A3B (MoE: 35B total params, 3B active) that unifies 6 AI orchestration capabilities into a single model.

Capabilities (6 modes)

Each training example has a mode tag prefix in the system prompt that teaches the model to activate different expert mixtures per task type.

Training Details

• Base model: Qwen3.5-35B-A3B (hybrid linear attention + Mamba SSM + 256 MoE experts)
• Method: LoRA (r=16, alpha=32, targets: q_proj + v_proj)
• Training data: 6,384 examples across 6 modes (SFT format)
• Epochs: 3 (1,077 steps)
• Best validation loss: 0.2504
• Training time: 12.9 hours on NVIDIA H100 80GB
• Precision: BF16 with BitsAndBytes 4-bit quantization (NF4)
• Optimizer: AdamW 8-bit, LR=2e-5 with cosine decay

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-35B-A3B", trust_remote_code=True)
model = PeftModel.from_pretrained(base_model, "hyperspaceai/thor-1-lora")
tokenizer = AutoTokenizer.from_pretrained("hyperspaceai/thor-1-lora")

prompt = "<|im_start|>system\n[INTENT] Classify the user's intent.\n<|im_end|>\n<|im_start|>user\nBook a flight to Tokyo next Friday<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Part of the Thor Ecosystem

THOR-1 is the unified brain of the Thor AI orchestration system, powering intelligent task routing, agent selection, and execution planning across 21 named agents and 1,200+ community agents.

Developed by: HyperspaceAI