Muse-Glimmer-30B-Fable-Distill

Muse-Glimmer-30B-Fable-Distill

meta-models/Muse-Glimmer-30B fine-tuned on agentic coding traces and chat distilled from Claude Fable 5, with a multilingual slice. Merged bf16 weights, drop-in for the base: same architecture (text decoder + vision tower), same tokenizer, same chat template, and the base's processor_config.json so AutoProcessor / image input work as in the base (verified after merging with an image request). Only the language model was trained.

The LoRA adapter this was merged from is published separately as armand0e/Muse-Glimmer-30B-Fable-Distill-LoRA.

Chat format and serving

Muse Glimmer uses its own harmony-style "Onyx ATEM" format — reasoning on a <|start|>assistant to=self<|message|>…<|eom|> channel, tool calls as <atem:function_calls> XML, tool results as <tool_output> blocks, <|eot|> end-of-turn. The chat template drives all of it (reasoning_strength: low/medium/high/xhigh, default high; tool-call arguments must be a dict, not a JSON string).

No stock vLLM (≤ 0.28) reasoning or tool parser matches this format — do not enable --reasoning-parser / --tool-call-parser with a stock name (gptoss, hermes, …), they will mis-parse. Serve without parsers (channel markers arrive in content) or register custom parser plugins. Requires transformers ≥ 5.15 (the muse_glimmer architecture ships in the library); with vLLM 0.26 use --model-impl transformers.

vllm serve armand0e/Muse-Glimmer-30B-Fable-Distill \
  --served-model-name muse-glimmer-fable \
  --model-impl transformers \
  --max-model-len 65536 --dtype bfloat16 \
  --override-generation-config '{"temperature": 1.0, "top_p": 0.95, "top_k": 64}'

Training data

651 examples / 3.62M tokens after rendering at a 24,576-token context (trim_followups; median 1.1k tokens, p90 20.4k). Prepared with teich 0.3.6 plus a format patch for this chat format. Reasoning kept and supervised. Rows without any assistant reasoning were rendered at reasoning_strength: low, rows with reasoning at high, so the strength flag in the system prompt actually predicts whether a reasoning channel follows (a first run rendered everything at high and learned to skip thinking). Harness text recorded as assistant turns in the raw sessions (API Error: …, Please run /login …, No response requested.) was removed before rendering — the first run memorised it verbatim. Supervision is scoped so the base's think-first routing survives: on rows without reasoning the initial routing header ( to=user<|message|> / to=<fn><|message|>) is left unsupervised and only the answer or <atem:function_calls> block is trained; on rows with reasoning the headers after <|eom|> are supervised, and the <|start|>assistant continuation that follows every <|eom|> is supervised so thinking → answer is learnable. (A run that supervised every routing header stopped opening the reasoning channel even at xhigh and called tools 1/4 vs the base's 4/4.)

Validated over every row before training: final answers supervised as to=user<|message|>…<|eot|>, reasoning as to=self<|message|>…<|eom|>, tool calls as recipient header + full <atem:function_calls> block + terminator; user/system/tool-output masked; 40% of tokens supervised.

Training

QLoRA on unsloth/Muse-Glimmer-30B-unsloth-bnb-4bit (NF4), LoRA r=32 / α=32 on the 416 text-decoder projections (self_attn.{q,k,v,o,gate}_proj, mlp.{gate,up,down}_proj; 209.6M parameters), merged into the bf16 base. 24,576-token context, LR 8e-5 linear, 2 epochs (164 steps), batch 1 × grad-accum 8, paged_adamw_8bit, max_grad_norm 0.3, single 64 GB GPU. Fused linear cross-entropy reproducing the model's output_multiplier pre-scale and tanh soft-cap (T=20), verified equal to the model's own loss. Base-model loss on this data at step 1: 1.17.

Limitations

Non-English is ~10% of tokens. Not benchmarked against an answers-only variant. The adapter was trained against NF4 base weights and merged into bf16 — standard QLoRA practice, but the merged model is not bit-identical to what training saw.