haan-v1-frozen

Haan v1 — frozen-backbone lineage

Haan is a Korean-first full-duplex speech model: a Qwen3-8B backbone reading Moshi-style multi-stream frames (Mimi codec, 12.5 Hz, 8 codebooks per audio stream) across four channels — native (inner) text, surface text, self audio, user audio. This repository preserves the frozen-backbone lineage: the pretrained spans are restored to their original weights and held, and only the interfaces around them learn.

Both checkpoints are bf16, from_pretrained-loadable (HaanForConditionalGeneration), 40 backbone layers.

Layout

How init/ was composed

Weight surgery over the full-training lineage's phase-2 checkpoint (RetentionLabs/haan-v1-full, phase2/) and the pristine assembly:

• layers 0–1: the trained checkpoint's first blocks
• layers 2–37: pristine Qwen3-8B, byte for byte — the span the run then froze
• layers 38–39: copies of the pristine last block
• depth decoder: pristine Moshi, except the input projections and role embeddings the run had fitted
• audio tables, role FiLM, text heads, text embedding above the vocabulary boundary: the trained checkpoint's

Training resumed at phase 2 with --freeze-layers 2:38 and the depth stack held (drift on the frozen spans measured 0.0).

Phase-2 selection

The run went to step 4000, milestone-scored every 200 steps. Step 1000 is where duplex control peaked before capacity drifted to the native channel:

Stream perplexity bottoms at ~1200 but the surface channel is already collapsing there; step 1000 is the balance point. Free-running duplex had not emerged by step 4000 (turn overlap ~0.99 throughout), so this checkpoint is preserved as the lineage's best state, not as a working duplex model.

Sibling repository

RetentionLabs/haan-v1-full holds the full-training lineage this one was composed from.