asteria-kimi-k27-brain-sparse

Asteria — Kimi K2.7 Brain (Sparse Slice)

Real brain weights from Kimi K2.7-Code (June 2026, latest Kimi model).

What's in this repo

• Source: moonshotai/Kimi-K2.7-Code
• Layers: 60 MoE layers (layers 1-60)
• Experts per layer: 8 (out of 384)
• Total experts: 480
• Per expert: 6 .bin files (weight_scale + weight_shape for gate/up/down)
• Total size: ~1.3 GB
• Format: BF16 (no squeezing, full precision scales)

What's NOT here (the other 99.79% of brain)

• weight_packed (the 4-bit weights, 7MB per expert) — fetch on-demand during inference
• Other 376 experts per layer — not loaded (sparse loading)
• Attention weights (MLA Q/K/V/O) — not loaded
• Token embedding (163,840 × 7,168) — not loaded
• Output projection (lm_head) — not loaded
• Layer 0 (dense, non-MoE) — not loaded

Verification

All weights are REAL trained Kimi K2.7 data:

• mean: ~0.006 (matches trained model)
• std: ~0.001 (matches trained model)
• NOT random init (random would have mean≈0, std≈1)

Usage

from huggingface_hub import snapshot_download
import torch, numpy as np

path = snapshot_download(
    repo_id="gamansai/asteria-kimi-k27-brain-sparse",
    allow_patterns="L01/E000/*"
)

raw = open(f"{path}/L01/E000/gate_proj_weight_scale.bin", "rb").read()
u16 = np.frombuffer(raw, dtype=np.uint16)
tensor = torch.from_numpy(u16.copy()).view(torch.bfloat16)
print(f"Shape: {tensor.shape}")
print(f"Mean: {tensor.float().mean():.6f}")  # ~0.007

Transfer details

• Transfer date: July 2026
• Transfer method: HTTP Range requests from HuggingFace CDN
• Speed: 8 parallel workers, ~0.9 experts/sec
• Total transfer time: ~25 minutes