Muse-Glimmer-30B-MXFP4-GGUF

Muse-Glimmer-30B MXFP4 GGUF

Base Model

Muse-Glimmer-30B by Meta — a 27.8B-parameter dense causal transformer with integrated vision encoder (ViT-G/14, ~1.8B params). Apache 2.0 licensed. Supports images, screenshots, charts, documents, and 131K+ context. No MTP heads — uses DFlash speculative decoding instead.

Vision Support

This model includes a full vision encoder (mmproj) embedded in the GGUF. Use llama-mtmd-cli or llama-server --mmproj for vision capabilities. The vision encoder uses dynamic-size input with patch size 14 and temporal patch 2.

Quantization

• Format: MXFP4 — OCP microscaled FP4 (E2M1 values with E8M0 power-of-two scaling per 32 values)
• BPW: 4.46 bits per weight
• Size: 14.8 GB
• Portable: MXFP4 works on any hardware with FP4 support (not Blackwell-only like NVFP4)

Tested Settings (llama.cpp build 1647+, RTX 5060 Ti 16GB)

llama-cli -m muse-glimmer-30b-mxfp4.gguf \
  -ngl 99 \
  --ctx-size 131072 \
  --flash-attn on \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --reasoning off \
  -t 8 --temp 0.7 --top-k 20 --top-p 0.95

Performance Results

Usage

llama.cpp CLI (text generation)

llama-cli -m muse-glimmer-30b-mxfp4.gguf -ngl 99 -c 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --reasoning off -p "Your prompt here"

llama.cpp Server (API)

llama-server -m muse-glimmer-30b-mxfp4.gguf -ngl 99 -c 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --reasoning off --host 0.0.0.0 --port 8080

Vision (requires llama-mtmd-cli)

llama-mtmd-cli -m muse-glimmer-30b-mxfp4.gguf -ngl 99 --flash-attn on --reasoning off --image /path/to/image.png -p "Describe this image"

Requirements

• llama.cpp build 1647+ with CUDA support (Muse-Glimmer architecture support merged Aug 2026)
• NVIDIA GPU with 16GB+ VRAM (tested on RTX 5060 Ti 16GB)
• Full GPU offload works — model fits entirely in VRAM
• --reasoning off recommended to suppress thinking token leakage

Notes

• Dense model (all 27.8B params active per token) — no --cpu-moe needed
• ~3x faster generation than NVFP4 variant (25.3 vs 8.7 t/s) — MXFP4 decode kernels are more efficient
• Sliding window attention (2048 for 75% of layers, full for 25%) keeps KV cache small at 128K
• For Blackwell-optimized variant, see the NVFP4 variant
• Reasoning cannot be fully disabled via config — use --reasoning off at inference

License

Apache 2.0 (inherited from base model)