Downloads · 30 days
14
9% of all-time downloads
AETHER-166B-A8B
B2J/AETHER-166B-A8B
AETHER-166B-A8B is a machine learning model from B2J. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
166B Mixture-of-Experts model with Wu Xing (Five Elements) oheng architecture.
All-time downloads
162
Public
Parameters
167B
340 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors340 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Apr 5, 2026
- Updated
- Apr 5, 2026
- SHA
- 7ba3d9eb
Base models
Model card
From the Hugging Face model README
AETHER-166B-A8B (Phase 0.5 — Oheng Calibrated)
166B Mixture-of-Experts model with Wu Xing (Five Elements) oheng architecture.
Model Details
- Parameters: 166B total, ~8B active per token
- Architecture: 25 layers, 5x5 MoE per layer, GDN+Mamba2+FullAttn hybrid
- Parent: Qwen3.5-397B-A17B (weight transplant + oheng calibration)
- Oheng: GenerateBoost (alpha[5]) + OvercomeGate
- Calibration: v11, 3000 steps, sparse KLD, best_kld=14.28 @ step 1489
- Precision: bfloat16
Training
- Phase 0: Weight transplant from Qwen3.5-397B → AETHER-166B
- Phase 0.5: Oheng calibration (offline sparse KLD distillation, 0.41% params)
- Phase 1: Full distillation (planned)
Status
This is a Phase 0.5 checkpoint. The model is NOT yet suitable for inference. Full distillation (Phase 1) is required for usable generation quality.
Hardware
Trained on 8x NVIDIA H200 (143GB each) with pipeline parallel.
- License
- apache-2.0
- Created
- Apr 5, 2026
- Updated
- Apr 5, 2026