Muse-Glimmer-30B-NVFP4

Muse Glimmer 30B NVFP4

Model ID: original-el8/Muse-Glimmer-30B-NVFP4
Base model: meta-models/Muse-Glimmer-30B
Created: August 11, 2026
Runtime status: qualified on one NVIDIA RTX PRO 6000 Blackwell GPU (SM120) with native vLLM Muse Glimmer support, the FlashInfer b12x NVFP4 kernel, piecewise CUDA graphs, and the Muse Glimmer DFlash assistant at speculative depth 16.
Quality result: on matched 140-question MMLU-Pro smoke tests, BF16/NVFP4 scored 109/109 with DFlash and 109/106 without DFlash; both passed all 16 paired core-workload canaries. Held-out WikiText-2 token perplexity was 5.6884 versus 5.4567 for BF16, a 4.25% increase.
Distribution fidelity: mean full-vocabulary next-token KL divergence of 0.091815 nats against BF16 across 24 matched contexts.
Container: ghcr.io/liquidgravityai/vllm-muse-glimmer:cu132-b12x-pr51655-a52f4d9c

Muse Glimmer 30B NVFP4 is an NVIDIA ModelOpt NVFP4 W4A4 conversion of the 29.8-billion-parameter Muse Glimmer vision-language model. The published safetensors files occupy 23,380,728,424 bytes (23.38 GB, 21.78 GiB); tokenizer and configuration files are additional.

Quantization specification

The safetensors index contains 2,684 tensors and 23,380,728,424 indexed bytes. Validation checked 1,572,733,760 scale values; all were finite and positive. All 104 packed Q/K/V and gate/up groups passed shared-global-scale equality. BF16-excluded tensors matched the source checkpoint byte-for-byte.

The checkpoint uses standard ModelOpt/Compressed-Tensors metadata in hf_quant_config.json. model-inputscales.safetensors stores one calibrated input scale for each quantized projection.

The complete deterministic first-$N$ selection rule, dataset hashes, template mode counts, per-source token counts, exact TOML configuration, and template revision are published under evaluation/. Calibration used the original Meta release template; the distributed checkpoint now carries Meta's corrected template, with both revisions and SHA-256 hashes recorded in evaluation/provenance.json.

Quantization and calibration attribution

The quantization and export pipeline is based on Luke Alonso's local-inference-lab/quant-toolkit, using revision 8bdb1016e52dd15f91e42d35c09096c0f31325f3. The toolkit provides the calibration runner, NVIDIA ModelOpt integration, resumable activation-statistics collection, fused-projection scale handling, and Hugging Face export path. The Muse Glimmer conversion adds its model adapter, calibration profile, quantization boundary, and validation gates. NVIDIA ModelOpt performs the calibration and NVFP4 conversion.

The toolkit's text-data builder declares the following public calibration source pool:

• agentic, coding, reasoning, math, and multilingual data: Orca AgentInstruct, CodeAlpaca, Self-OSS-Instruct, StarCoderData, Hermes Function Calling, Glaive Function Calling, OpenOrca, GSM8K, MathInstruct, Alpaca Chinese, COIG-CQIA, and Zhihu-KOL
• diverse conversations: WildChat-nontoxic and LMSYS-Chat-1M
• multimodal images: COCO val2017, selected and paired with calibration prompts by the toolkit's tools/build_mm_calib.py

The normalized text JSONL files retain message content but not a per-record upstream source identifier. The manifests under evaluation/ identify and verify the exact local files and first-$N$ selections with SHA-256 hashes, but they do not recover record-level provenance. Each upstream dataset remains subject to its own license and terms.

Quality validation

MMLU-Pro

BF16 and NVFP4 were evaluated twice on the same 140-question, five-shot, high-reasoning MMLU-Pro slate. Both comparisons use the same final-answer extraction rule for BF16 and NVFP4.

The DFlash run had 100 both-correct, 9 BF16-only, 9 NVFP4-only, and 22 both-incorrect outcomes (exact two-sided McNemar $p=1.0$). The no-DFlash run had 93 both correct, 16 BF16-only, 13 NVFP4-only, and 18 both incorrect ($p=0.711$).

The historical runners had stored 106/140 for BF16 and 108/140 for NVFP4 because they used inconsistent extraction behavior; applying the shared rule to both unchanged output sets produces the DFlash tie above. The no-DFlash comparison removes the assistant-model confound and uses Meta's corrected chat template, but its -2.14-point observed difference means this small slate does not support a tight non-inferiority claim. Neither result establishes statistical equivalence or superiority.

The exact question IDs, rendered prompts, per-request seeds and sampling parameters, all four output sets, shared scorer, replay tool, immutable no-DFlash runtime digest, and paired summaries are published in evaluation/mmlu-pro/.

Core-workload canaries

BF16 and NVFP4 both passed all 16 deterministic no-DFlash canaries:

The paired comparator verified that every request was identical except for the served model name. These are targeted behavior canaries, not broad benchmark estimates. Protocols, generated images, complete long-context prompts, requests, responses, hashes, runner, and comparator are published in evaluation/core-workloads/.

Held-out token perplexity

BF16 and NVFP4 were measured without speculative decoding on the WikiText-2 raw test split. The evaluator concatenated all 4,358 rows with double newlines, tokenized once with the Muse Glimmer tokenizer, and scored each of 289,691 non-BOS tokens exactly once using 4,096-token windows and a 2,048-token stride. Overlapping prefixes supplied context only; raw token IDs bypassed the chat template.

NVFP4 increased perplexity by 4.2459% relative to BF16. A 100,000-sample sequential 2,048-token block bootstrap placed the PPL ratio at 1.0399–1.0452 for its central 95% interval. This misses the provisional no-more-than-2% relative-regression gate; the checkpoint is therefore not qualified against that strict PPL threshold.

Token perplexity depends on the tokenizer, corpus construction, and windowing protocol. Compare these values only under this matched setup. The pinned dataset revision and hashes, per-checkpoint summaries, all 289,691 paired raw token log-probabilities, runner, and paired comparator are published in evaluation/perplexity/.

Full-vocabulary next-token fidelity

The fidelity measurement compares exact next-token distributions from BF16 and this NVFP4 checkpoint.

Conditions

• Hardware: one NVIDIA RTX PRO 6000 Blackwell GPU (SM120).
• Runtime: the same native vLLM Muse Glimmer implementation for both checkpoints.
• Protocol: 12 fixed bare-completion prompts with an Answer: cue, each measured before generation and after eight tokens along the same fixed NVFP4 greedy trajectory.
• Vocabulary: all 202,048 output tokens; no top-$k$ approximation.
• Direction: $D_{KL}(P_{BF16}\parallel P_{NVFP4})$ using natural logarithms.
• Computation: raw log probabilities renormalized in float64.

The maximum-divergence context was the coding prompt at trajectory depth 8, where the checkpoints selected different top tokens. These values measure token-distribution fidelity, not downstream task accuracy.

The exact contexts, raw [24, 202048] BF16 and NVFP4 arrays, per-context results, and independent NumPy verifier are published in evaluation/kl-fidelity/.

Qualified runtime

A compatible runtime must provide all of the following:

1. Native Muse Glimmer model, processor, reasoning-parser, and tool-parser support.
2. NVIDIA ModelOpt NVFP4 checkpoint loading.
3. An NVFP4 linear kernel for the target GPU. The qualified SM120 runtime used FlashInfer b12x.
4. Transformers 5.15.0 or another release with MuseGlimmerForConditionalGeneration.
5. The Muse Glimmer assistant checkpoint for DFlash speculative decoding, when DFlash is enabled.

The qualified target-plus-drafter process loaded 27.1 GiB of weights. At --gpu-memory-utilization 0.94, it allocated a 1,963,206-token GPU KV cache and reported 14.98× maximum concurrency at 131,072 tokens. DFlash and a 131K context budget are not claimed to fit on 24 GB or 32 GB GPUs. Remove DFlash and reduce context/concurrency for lower-memory systems, then qualify that configuration independently.

The production command is:

vllm serve original-el8/Muse-Glimmer-30B-NVFP4 \
  --host 0.0.0.0 \
  --port 8000 \
  --linear-backend flashinfer_b12x \
  --attention-backend FLASHINFER \
  --max-model-len 131072 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 16384 \
  --gpu-memory-utilization 0.94 \
  --enable-prefix-caching \
  --enable-chunked-prefill \
  --enable-auto-tool-choice \
  --tool-call-parser muse_glimmer \
  --reasoning-parser muse_glimmer \
  --speculative-config '{"model":"meta-models/Muse-Glimmer-30B-assistant","method":"dflash","num_speculative_tokens":16,"draft_sample_method":"probabilistic"}'

The b12x backend is selected explicitly. Do not assume that auto selects the same kernel.

Text request

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Muse-Glimmer-30B-NVFP4",
    "messages": [
      {"role": "user", "content": "Explain why idempotency matters for payment APIs."}
    ],
    "temperature": 0,
    "max_tokens": 256
  }'

Muse Glimmer accepts OpenAI-style multimodal message content. Supply an image_url content part before the text content part in the same user message.

Published GHCR image

The Linux/amd64 image contains the qualified CUDA 13.2.1 runtime, the pinned b12x/FlashInfer stack, Transformers 5.15.0, the integrated Muse Glimmer model and parsers, and the DFlash fixes used for the validation above.

Docker

The assistant repository is gated. Export a Hugging Face token with access before starting the DFlash configuration.

export HF_TOKEN=hf_...

docker pull \
  ghcr.io/liquidgravityai/vllm-muse-glimmer@sha256:34173f7bc4f6a5a1e590a899ef4753b22e7d87e18b156f74958ec3b2dbf74871

docker run --rm --gpus all --ipc=host -p 8000:8000 \
  -e HF_TOKEN \
  -v "${HF_HOME:-$HOME/.cache/huggingface}:/root/.cache/huggingface" \
  ghcr.io/liquidgravityai/vllm-muse-glimmer@sha256:34173f7bc4f6a5a1e590a899ef4753b22e7d87e18b156f74958ec3b2dbf74871 \
  vllm serve original-el8/Muse-Glimmer-30B-NVFP4 \
    --host 0.0.0.0 \
    --port 8000 \
    --linear-backend flashinfer_b12x \
    --attention-backend FLASHINFER \
    --max-model-len 131072 \
    --max-num-seqs 8 \
    --max-num-batched-tokens 16384 \
    --gpu-memory-utilization 0.94 \
    --enable-prefix-caching \
    --enable-chunked-prefill \
    --enable-auto-tool-choice \
    --tool-call-parser muse_glimmer \
    --reasoning-parser muse_glimmer \
    --speculative-config '{"model":"meta-models/Muse-Glimmer-30B-assistant","method":"dflash","num_speculative_tokens":16,"draft_sample_method":"probabilistic"}'

Docker Compose

Save the following as compose.yaml:

services:
  muse-glimmer:
    image: ghcr.io/liquidgravityai/vllm-muse-glimmer@sha256:34173f7bc4f6a5a1e590a899ef4753b22e7d87e18b156f74958ec3b2dbf74871
    gpus: all
    ipc: host
    shm_size: 16gb
    ports:
      - "8000:8000"
    environment:
      HF_TOKEN: ${HF_TOKEN:?Set HF_TOKEN}
    volumes:
      - ${HF_HOME:-./hf-cache}:/root/.cache/huggingface
    command:
      - vllm
      - serve
      - original-el8/Muse-Glimmer-30B-NVFP4
      - --host
      - 0.0.0.0
      - --port
      - "8000"
      - --linear-backend
      - flashinfer_b12x
      - --attention-backend
      - FLASHINFER
      - --max-model-len
      - "131072"
      - --max-num-seqs
      - "8"
      - --max-num-batched-tokens
      - "16384"
      - --gpu-memory-utilization
      - "0.94"
      - --enable-prefix-caching
      - --enable-chunked-prefill
      - --enable-auto-tool-choice
      - --tool-call-parser
      - muse_glimmer
      - --reasoning-parser
      - muse_glimmer
      - --speculative-config
      - '{"model":"meta-models/Muse-Glimmer-30B-assistant","method":"dflash","num_speculative_tokens":16,"draft_sample_method":"probabilistic"}'

Then run:

docker compose up -d
docker compose ps
curl --fail http://127.0.0.1:8000/health

vLLM implementation provenance

This image is not a stock vLLM wheel. It overlays the validated Muse Glimmer sources onto a pinned CUDA 13.2 b12x base. The relevant pull requests are listed explicitly:

Local compatibility changes preserve the Muse assistant's encoder.fc and encoder.output_norm_enc checkpoint names with the pinned base loader, retain the validated SM120 vision-parallel API, and keep b12x explicit rather than automatic.

Runtime verification of the published image

The immutable image above was loaded with this checkpoint and meta-models/Muse-Glimmer-30B-assistant on one RTX PRO 6000 Blackwell GPU. Verification observed:

• healthy service startup with zero container restarts;
• FlashInferB12xNvFp4LinearKernel selected for NVFP4 GEMM;
• piecewise CUDA graph capture and DFlash speculative depth 16;
• deterministic text generation with final content MUSE GLIMMER READY;
• a valid get_weather({"city":"Paris"}) tool call through the Muse parser;
• successful image preprocessing and a correct one-sentence description of the test scene;
• the same Tetris generation protocol used for the earlier baseline completed 4,501 output tokens in 32.876 seconds (136.91 end-to-end output tokens/s); because sampling is stochastic, this is a runtime check rather than a controlled performance claim;
• eight concurrent 512-token requests all completed, producing 4,096 tokens in 7.362 seconds, after which the service remained healthy with zero restarts.

Limitations

• The published container is qualified for Linux/amd64 and NVIDIA SM120. A one-DGX-Spark, TP=1 SM121 canary loaded the checkpoint and selected both b12x and CUTLASS NVFP4 paths, but the available ARM64 runtime failed its profile run with CUDA error: no kernel image is available for execution on the device. SM121 is therefore not qualified; exact attempts are recorded in evaluation/single-spark-canary.json.
• ModelOpt NVFP4 checkpoint support in vLLM is experimental and its serialization contract can change.
• The DFlash assistant increases memory use and requires access to the gated assistant repository.
• Quantization can change rare-token probabilities and downstream behavior even when aggregate benchmarks pass.
• Validate task-specific accuracy, long-context behavior, safety, and tool permissions before deployment.

License and use

The base model and this quantized checkpoint are distributed under the Apache License 2.0. Review the base model card for architecture, intended use, limitations, and safety information.