Muse-Glimmer-30B-Abliterated-Aggressive-NVFP4

Muse-Glimmer-30B-Abliterated-Aggressive — NVFP4 (lean, vision-preserved)

A lean 22 GB NVFP4 (W4A4) quant of jorkle/Muse-Glimmer-30B-Abliterated-Aggressive (itself an aggressively-decensored — KL-conserving LoRA-SFT — build of Meta's meta-models/Muse-Glimmer-30B).

Built to run a decensored, vision-preserving Muse-Glimmer on a single ~24–32 GB Blackwell / DGX-Spark (GB10) in vLLM. At the time of quanting, no decensored lean NVFP4 Muse existed — the public lean NVFP4s were the censored base, and the decensored builds were only GGUF / bf16 / MLX / a fat 28 GB NVFP4. This fills that gap.

What it is

• Format: compressed-tensors NVFP4, W4A4 group-16, via Intel AutoRound (scheme="NVFP4", dataset="NeelNanda/pile-10k", nsamples=128, seqlen=2048, iters=200, quant_nontext_module=False).
• Vision tower + lm_head kept in BF16 (quant_nontext_module=False) — pristine multimodal input at zero decode cost; only the decoder Linear layers (the size + per-token bandwidth) are quantized to FP4.
• ~22 GB on disk (vs the ~28 GB fat abliterated NVFP4). Multimodal (perception encoder) intact.

Serving (vLLM)

Needs a Muse-Glimmer-capable vLLM build (e.g. vllm/vllm-openai:muse-glimmer):

vllm serve <this-model> --served-model-name muse-aggressive \
  --reasoning-parser muse_glimmer --enable-auto-tool-choice --tool-call-parser muse_glimmer \
  --trust-remote-code --max-model-len 131072 --kv-cache-dtype fp8

Quantization auto-detects (compressed-tensors) — no --quantization flag needed. Use a Reasoning strength: low system line if you want content in content rather than reasoning_content.

Measured on GB10 (DGX-Spark, 224 GB/s, single-stream)

Honest notes

• This is the aggressive decensor. Its KL-LoRA-SFT decensoring lowers instruction-following (IFEval .684) vs a plain weight-edit abliteration of the same base (which measured IFEval ~.90). Prefer this build for maximally-permissive / RP use; for instruction-following-critical work a manual-abliterated quant is better.
• Speed is bandwidth-bound (dense 30B ÷ ~224 GB/s). No speculative drafter is included — pairing a matched DFlash/EAGLE drafter would roughly double decode.

Attribution

Base: Meta Muse-Glimmer-30B. Decensoring: jorkle (Abliterated-Aggressive, KL-LoRA-SFT). NVFP4 quant: this repo (AutoRound, compressed-tensors). Inherits the upstream Muse-Glimmer license.