celestia-1.4b-moe

Celestia-1.4B-MoE (Creative Workspace Frontier Model)

Celestia is a domain-specific foundation model designed to orchestrate creative multi-mode workflows in Lenout (2D Drawing, Design, Whiteboard, Flowcharts, Presentations, Motion Video, and 3D Studio).

Architecture Highlights

• Multi-Head Latent Attention (MLA): DeepSeek-style KV compression reducing cache footprint by 70%+.
• Sparse DeepSeekMoE: 8 fine-grained routed experts + 1 shared expert (2 active experts per token).
• Deep Reasoning: Native support for Chain-of-Thought planning in <think>...</think> tags via GRPO Reinforcement Learning.
• Lenout Action DSL: Native structured JSON output within <action_start>...</action_end>.

Usage Example

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "alenout/celestia-1.4b-moe"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True, torch_dtype="bfloat16")

prompt = "<system>\nTarget Mode: design\n</system>\n<user>\nสร้างการ์ดสินค้า 3 ใบแนวนอน\n</user>\n<assistant>\n<think>"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0]))

Citation

@misc{alenout2026celestia,
  author = {Alenout AI Team},
  title = {Celestia: Specialized Creative Workspace Foundation Model},
  year = {2026},
  url = {https://github.com/alenout/celestia}
}