qwen3.6-27b-nla-av

Qwen3.6-27B — NLA Activation Verbalizer (AV), all-module

A Natural Language Autoencoder verbalizer for Qwen/Qwen3.6-27B: given a hidden activation, the model generates a natural-language description of what that activation represents. Trained with the EasyNLA / nanoNLA recipe (warm-start SFT on ceselder/qwen3-8b-nla-L24-finefineweb-100k explanations + GRPO RL against a reconstruction reward).

• Extraction layer: 42 (of 64). Injection layer: 1 (Karvonen norm-matched).
• Adapter scope: all-module LoRA (attn + MLP + deltanet), merged into bf16 here — self-contained, no PEFT/LoRA loading needed.
• Held-out FVE ≈ 75% (fraction of activation variance explained by the reconstruction of the generated text).

Inference (full rollout on one activation)

Use scripts/show_nla_generations.py from EasyNLA (github.com/asherps/EasyNLA):

python scripts/show_nla_generations.py \
  --base-ckpt ceselder/qwen3.6-27b-nla-av \   # THIS merged model (already includes the verbalizer)
  --av-lora   ceselder/qwen3.6-27b-nla-av \   # same path: merged model has no separate LoRA
  --sidecar   nla_meta.yaml \
  --parquet   example_activations.parquet \
  --max-new-tokens 200

Core loop (≈10 lines): load model → register_karvonen_hook(...injection_token_id..., layer_idx=1) → set the activation into the hook's vref[0] → build the prompt with the injection char (㈜) at the marker → model.generate(). The injection contract (token id, neighbors, scale) is in nla_meta.yaml — read it with nla.config.load_nla_config.

Note: because this is merged, load it directly as the base. If instead you use the raw Qwen/Qwen3.6-27B + a separate LoRA, the base MUST be the matching merged warm-start, not the raw model, or you drop the warm-start and get garbage.

example_activations.parquet holds a few rows (prompt + activation vector) to test end-to-end.