Muse-Glimmer-30B-UD-Q4_K_XL-262k-patch

Muse Glimmer 30B — 262k Context Patch

This repo provides a patch to extend the Muse Glimmer 30B GGUF model's context window from 131072 (131k) to 262144 (262k) tokens.

The patch edits 4 bytes in the GGUF file header (the context_length uint32 field). No model weights are modified.

Why this works

Muse-Glimmer's architecture supports longer context:

• 13 full_attention layers with NoPE (rope_theta=0) — these attend across the entire sequence without positional encoding. No change needed.
• 39 sliding_attention layers with rope_theta=500000 and sliding_window=2048 — these are self-contained at 2048 tokens and don't grow with context length.

Only the GGUF metadata cap needs to be raised. The model's actual behavior is determined by which layer is attending at each position, and only the 13 NoPE layers ever need to see positions beyond 2048 — and they don't use position info at all.

What does NOT work (and why)

• CLI flags like -c 262144 — LM Studio passes them to llama-server but clamps based on the GGUF header before the flag is honored.
• Config JSON keys like llm.load.contextLength — same clamp applies.
• RoPE scaling fields — LM Studio silently ignores them for llama.cpp backend.
• No regression at 131k — the model was fine-tuned at 131072, so positions 0–131071 are in-distribution. No quality degradation in that range.

Usage

Prerequisites

• A Muse Glimmer 30B GGUF file (e.g., Muse-Glimmer-30B-UD-Q4_K_XL.gguf from unsloth/Muse-Glimmer-30B-GGUF)
• Python 3.6+

Apply the patch

python3 muse-262k-patch.py /path/to/Muse-Glimmer-30B-UD-Q4_K_XL.gguf

Expected output:

Found muse-glimmer.context_length at offset 281, current=131072
Backed up header to /path/to/Muse-Glimmer-30B-UD-Q4_K_XL.gguf.header.bak
Patched to 262144, verified=262144

Load in LM Studio

lms load muse-glimmer-30b -y

Verify with lms ps — context should show 262144.

VRAM requirements (RTX 4090 24GB)

Muse-Glimmer-30B Q4_K_XL GGUF (15 GB) on RTX 4090:

• Model weights: ~15 GB
• Runtime overhead: ~1 GB
• KV cache at 262k (f16): ~1.3 GB
• Total: ~17.3 GB used, ~7 GB headroom

Empirical per-token KV cost is much lower than naive calculations because sliding-window layers are capped at 2048 tokens. Only the 13 full-attention layers grow with context length.

Maximum context on RTX 4090 with nothing else loaded: ~500k tokens at f16, up to ~1M if using q4 KV cache (requires llama-server direct).

Durability

The patch persists across:

• LM Studio restarts
• System reboots

The patch does NOT survive:

• Re-downloading the GGUF (a fresh download overwrites the patched file with 131072)

If your file gets re-downloaded, simply re-run muse-262k-patch.py on the new file. The script finds the offset dynamically.

Backup

The original 4096-byte header is saved to <file>.header.bak next to the GGUF. Restore with:

dd if=original.gguf.header.bak of=original.gguf bs=1 count=4096 conv=notrunc

Related repos

• Original GGUF: unsloth/Muse-Glimmer-30B-GGUF
• Original model: meta-models/Muse-Glimmer-30B

License

Inherits from the upstream Muse-Glimmer-30B license. Check meta-models/Muse-Glimmer-30B for terms before use.