Muse-Glimmer-30B-NVFP4-GGUF

Muse-Glimmer-30B NVFP4 GGUF

Base Model

Muse-Glimmer-30B by Meta — a 27.8B-parameter dense causal transformer with integrated vision encoder (ViT-G/14, ~1.8B params). Apache 2.0 licensed. Supports images, screenshots, charts, documents, and 131K+ context. No MTP heads — uses DFlash speculative decoding instead.

Vision Support

This model includes a full vision encoder (mmproj) embedded in the GGUF. Use llama-mtmd-cli or llama-server --mmproj for vision capabilities. The vision encoder uses dynamic-size input with patch size 14 and temporal patch 2.

Quantization

• Format: NVIDIA FP4 (NVFP4) — 4-bit E2M1 values with E4M3 scaling per 16 values + FP32 tensor scale
• BPW: 4.69 bits per weight
• Size: 15.6 GB
• Target hardware: NVIDIA Blackwell GPUs (RTX 50 series) — requires CUDA kernels at inference

Tested Settings (llama.cpp build 1647+, RTX 5060 Ti 16GB)

llama-cli -m muse-glimmer-30b-nvfp4.gguf \
  -ngl 99 \
  --ctx-size 131072 \
  --flash-attn on \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --reasoning off \
  -t 8 --temp 0.7 --top-k 20 --top-p 0.95

Performance Results

Usage

llama.cpp CLI (text generation)

llama-cli -m muse-glimmer-30b-nvfp4.gguf -ngl 99 -c 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --reasoning off -p "Your prompt here"

llama.cpp Server (API)

llama-server -m muse-glimmer-30b-nvfp4.gguf -ngl 99 -c 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --reasoning off --host 0.0.0.0 --port 8080

Vision (requires llama-mtmd-cli)

llama-mtmd-cli -m muse-glimmer-30b-nvfp4.gguf -ngl 99 --flash-attn on --reasoning off --image /path/to/image.png -p "Describe this image"

Requirements

• llama.cpp build 1647+ with CUDA support (Muse-Glimmer architecture support merged Aug 2026)
• NVIDIA GPU with 16GB+ VRAM (tested on RTX 5060 Ti 16GB)
• Full GPU offload works — model fits entirely in VRAM
• --reasoning off recommended to suppress thinking token leakage

Notes

• Dense model (all 27.8B params active per token) — no --cpu-moe needed
• Sliding window attention (2048 for 75% of layers, full for 25%) keeps KV cache small at 128K
• For faster generation (~3x), see the MXFP4 variant
• Reasoning cannot be fully disabled via config — use --reasoning off at inference

License

Apache 2.0 (inherited from base model)