Muse-Glimmer-30B-tq4-g64

Muse-Glimmer-30B-tq4-g64

4-bit TurboQuant build of meta-models/Muse-Glimmer-30B for Apple Silicon / MLX.

14.88 GiB, and it scores better than the affine 4-bit build that is 5 GB larger.

Documentation: manjunathshiva.github.io/turboquant-mlx: the Muse Glimmer page, getting started, choosing a model for your Mac's memory, guides for serving an OpenAI-compatible API, local coding agents, vision models and memory and macOS settings.

Why this build exists

800-token fixed-passage perplexity, greedy, Apple M4 Max 64 GB, same harness for every row. Lower is better.

Two honest notes on that table:

• Decode is 2.4–2.8× slower than affine 4-bit. TurboQuant decodes a Lloyd-Max codebook and applies an online Hadamard rotation; MLX's affine path is simpler and faster. If you want maximum tokens/sec and have the memory, use the mlx-community build. This build is for when the 5 GB matters.
• The size win is not magic. The affine build leaves embed_tokens and the entire 50-layer vision tower in bf16 — 3.3B parameters, most of its extra 5 GB. This build quantizes them.

Install

pip install "turboquant-mlx-full[vlm]>=0.21.1"

That is the whole install. [vlm] pulls mlx-vlm >= 0.6.12, the first release carrying the muse_glimmer model classes (PR #1838, merged 2026-08-10).

Following older instructions? They pinned mlx-vlm to a git commit and forced transformers==5.15.0. Both are obsolete, and the git pin is now actively wrong: the merged #1838 moved the embedding norm out of NormedEmbedding, and turboquant-mlx < 0.21.0 crashes against it with AttributeError: module '...muse_glimmer.language' has no attribute 'NormedEmbedding'. Use >= 0.21.1 and drop every pin.

Sanity-check before downloading 15 GB:

python -c "
import turboquant_mlx; print(turboquant_mlx.__version__)
from mlx_vlm.models import muse_glimmer; print('muse_glimmer OK')"

Usage

python -m turboquant_mlx.generate_vlm \
  --model manjunathshiva/Muse-Glimmer-30B-tq4-g64 \
  --prompt "Explain what a Hadamard rotation does to a weight distribution." \
  --max-tokens 512

# with an image
python -m turboquant_mlx.generate_vlm \
  --model manjunathshiva/Muse-Glimmer-30B-tq4-g64 \
  --image chart.png --prompt "What does this chart show?" --max-tokens 512

Muse Glimmer emits a reasoning channel before its answer. Pass --reasoning low|medium|high|xhigh to control how long it deliberates — the default is high, which spends hundreds of tokens before a short answer and is most of the wall clock on a slow machine. --no-think asks for the least it supports (there is no "off" level, so it still thinks briefly). Meta recommends temperature 1.0, top_p 0.95, top_k 64.

Serve it (OpenAI-compatible)

turboquant-serve-vlm \
  --model manjunathshiva/Muse-Glimmer-30B-tq4-g64 \
  --reasoning-strength low --port 8080

turboquant-serve wraps mlx_lm.server, which cannot load multimodal architectures — use turboquant-serve-vlm (0.21.1+), which drives mlx-vlm's server: OpenAI and Anthropic routes, per-model tool parsers, continuous batching.

Tool calling works — finish_reason: tool_calls with correct name and arguments, and tool-result round trips answer correctly. Two Muse-specific problems are handled for you, and both fail silently if you serve this model any other way:

• The reasoning channel is kept out of content. Muse answers in a harmony-style channel format, and the ATEM tool parser only strips that envelope when a tool call was parsed. So tool turns look fine while an ordinary turn hands the caller the model's private deliberation as its reply. Here it goes to reasoning_content, where it belongs.
• reasoning_effort actually reaches the template. OpenAI clients send reasoning_effort; this template only reads reasoning_strength, and otherwise deliberates at its high default. Requests are translated, and --reasoning-strength sets the default for clients that send nothing — which is most agent harnesses. Measured: a tool-result turn cost 54 completion tokens at low versus 106 at high, for the same answer.

Agentic coding & vision — measured

OpenCode: 3/3 pass. Task: a repo with a planted off-by-one in average() and a failing pytest suite; the prompt gives the exact venv command and asks to run → find → fix → re-run. Every run executed the given command first try, made the correct minimal edit (sum(values) / len(values)), and left the test file byte-identical to a fresh reference — a pass here means the bug was really fixed, not that the tests were edited until they agreed.

No run drifted: action counts stayed in a tight band, with none of the cross-turn perseveration that sinks smaller/harsher quantizations in agent harnesses. medium passes too but buys nothing measurable — stay on low.

Vision: 4/4, against mlx-community/Muse-Glimmer-30B-4bit as a control at 4/4. OCR of rendered text, counting with distractor shapes, reading which bar of a chart is tallest, and naming the shape in the top-left — all correct, ~11–13 s per image.

That control matters: this build quantizes embed_tokens and the entire 50-layer ViT-G/14 vision tower, which the affine 4-bit build leaves in bf16. Compressing them cost nothing measurable on these cases. Caveat: 4/4 vs 4/4 means no regression detected, not identical vision quality — these are clean synthetic images with unambiguous answers, not a vision benchmark.

Fixed in 0.21.1 — use it for agents. On 0.21.0 the model completes the work but never emits a closing summary, and every run ends at the passing test. That was a serving bug, not the model: ATEM's tool_call_start is to=self<|message|>, the same string Muse Glimmer opens every turn with, so mlx-vlm's streaming content suppressor latched on the first reasoning token and dropped everything after it. Any client that declares tools saw an empty final message. Re-verified after the fix: prose after the passing test goes 0 → 6 lines on both builds, with tool calls, action counts and the test-file md5 unchanged.

Full method, per-run numbers and reproduce steps: docs/RESULTS_MUSE.md.

Will it fit my Mac?

Real turboquant-plan output for this model, on an M4 Max 64 GB:

Model
  type                 muse_glimmer  (turboquant 4-bit g64)
  weights (exact)      15.97 GB

Machine
  Metal working set    55.66 GB   ← the real ceiling
  system RAM           68.72 GB

Projection at 16,384 tokens of context
  weights              15.97 GB
  KV cache             0.30 GB  (17.9 KB/token, hybrid: 13/52 full-attention, 39 sliding (window 2048))
  prefill workspace    6.70 GB  (estimate, at --prefill-step-size 2048)
  runtime reserve      1.00 GB  (buffer cache, activations, fragmentation)
                       ----------------------------------
  peak                 23.97 GB of 61.85 GB usable   37.88 GB headroom

Verdict: ✅ RESIDENT — fits fully in memory

Check your own machine before downloading 15 GB:

turboquant-plan --model manjunathshiva/Muse-Glimmer-30B-tq4-g64 --ram-gb 32

The KV cache is unusually cheap here — 17.9 KB/token, 0.30 GB at 16K context — because 39 of 52 layers use a 2048-token sliding window. Long context costs far less than the parameter count suggests.

32 GB Macs: comfortable. 24 GB: workable with a smaller --prefill-step-size. 16 GB: use tq3-g64 instead — it is measured running resident on a 16 GB Mac mini, this build is 2 GB too large for that.

How it was made

python -m turboquant_mlx.convert_vlm \
  --hf-path meta-models/Muse-Glimmer-30B \
  --mlx-path ./Muse-Glimmer-30B-tq4-g64 \
  --bits 4 --group-size 64 --quantize-extras --extras-bits 4

Data-free: randomized Hadamard rotation to Gaussianize the weights, per-group RMS scaling, then nearest Lloyd-Max centroid. No calibration set. The transformer linears are 4-bit polar codebook (4.25 bpw); lm_head, the embedding and the vision tower are 4-bit affine.

lm_head is deliberately affine, not polar. At 202048 × 6656 = 1.345B parameters it is 10× the largest MLP matrix, and TurboQuant's dense prefill path would materialize it — costing ~19 GB of transient peak to save 0.21 GB on disk.

Limitations

• Slower decode than affine 4-bit (9.4 vs 26.1 tok/s). Known and expected.
• Needs mlx-vlm >= 0.6.12 and turboquant-mlx >= 0.21.1; older turboquant-mlx crashes against the merged mlx-vlm.
• Perplexity is a single 800-token passage on one domain — a size/quality sanity check, not a broad evaluation. Meta's published benchmark numbers are for the bf16 model and have not been re-measured here.
• The vision tower is quantized to 4-bit affine and its effect on fine-grained visual tasks (OCR, ScreenSpot-style grounding) has not been measured.
• No agentic/tool-calling validation has been run on this build.

Licence

Apache-2.0, inherited from the base model. LICENSE and Meta's USAGE_POLICY.md are included in this repo; the usage policy applies to derivatives.