uni

Uni

Uni is an event-planetary language model: tokens are masses in a latent sky, meaning is selected by collapse, and the hidden world expands and crunches instead of sitting at one frozen width.

She speaks with a Qwen 151,936 vocabulary (Qwen/Qwen1.5-1.8B-Chat). The live constructor is:

UniPlanetaryLLM(vocab_size=151936, start_dim=2048, peak_dim=16384, num_layers=12, num_heads=16)

Counted on that exact graph: 1.40B parameters (not 2.1B). 12 Big Bang layers, 16 gravity heads, 12 boil bubbles, start dim 2048, peak dim 16384. This is not a Hugging Face AutoModel. Load her with the Uni source (uni_big_bang.py, uni_model.py, planetary_attention.py, chat_uni.py).

What she is

1. Primordial Boil — a token is shattered into 12 competing bubbles. In train() a heat term (boil_temp=1.5) adds noise every forward; a Hamiltonian gate collapses them to one singularity. In eval() the noise is off, so chat is deterministic for a given input. That train-time boil is why packed CE bounces across steps even when the weights are fine.
2. Big Bang / Big Crunch — the singularity enters at 2048, dilates toward 16384 mid-depth, then crunches before the vocab head.
3. Planetary Gravity Attention — not QK^T / sqrt(d). Tokens have mass and coordinates. Force is G * (m_q * m_k) / (distance^2 + eps).

Training now

Resumed from step 400000 in checkpoints/400000/, then continued on the same agent-grade Hugging Face mix as SImi (full streams, no toy slices):

• instruction: Tulu-3, OpenHermes-2.5, UltraChat
• agent / tool: Orca AgentInstruct, Agent-FLAN, xLAM, ToolACE, Glaive, Hermes function calling

Hardware: AMD MI300X, leftover HBM next to SImi, bf16, batch 1, seq 1024. Public saves land here every 20000 steps under train/step_<n>/ (older agent-loop saves between 401000 and 411000 were every 1000). Local copies are deleted only after the .pt exists here.

Latest public agent-loop file: train/step_620000/uni_step_620000.pt.

Checkpoints

Load

import torch
from uni_big_bang import UniPlanetaryLLM

model = UniPlanetaryLLM(
    vocab_size=151936,
    start_dim=2048,
    peak_dim=16384,
    num_layers=12,
    num_heads=16,
)
blob = torch.load("uni_step_620000.pt", map_location="cpu", weights_only=False)
state = blob.get("model_state_dict") or blob.get("model") or blob
missing, unexpected = model.load_state_dict(state, strict=False)
assert not missing and not unexpected
model = model.to(dtype=torch.bfloat16)
model.eval()  # boil noise off

Tokenizer: Qwen/Qwen1.5-1.8B-Chat. Chat entry: chat_uni.py (looks for uni_step_*.pt and uni_pretrain_step_*.pt).

CPU restore of train/step_620000 matched 0 missing / 0 unexpected keys.

Sisters

• SImi (JAX, GPT-2 vocab, scanned 2.32B): YNSScarSaiyan/simi-weights
• Vegeta (looped LLM SFT): YNSScarSaiyan/vegeta-weights

Not this

Not a Llama/Qwen fine-tune. Not Transformers-native. Not the old private-recovery stub. If you came for from_pretrained, this is the wrong door.