Molmo2-8B-GGUF

Molmo2-8B GGUF

GGUF conversion of allenai/Molmo2-8B, published by Mu2 Solutions.

Molmo2-8B is Allen Institute for AI's fully open multimodal model (Apache 2.0): SigLIP vision encoder plus a pooling cross-attention projector over a Qwen3-8B text backbone (qwen3 architecture).

License

Apache 2.0, same as the source model. No additional restrictions.

Files

Q4_K_M is the recommended default. Pair any text quant with the mmproj for vision.

The projector was replaced on 2026-09-28. The earlier Molmo2-8B-mmproj-F16.gguf (943,332,544 bytes) stored its pooling tensors in a fused mm.pool.kv layout and omitted the mm.image_patch_add row, so current Molmo2 support in llama.cpp refuses it at load with unable to find tensor mm.pool.k.weight. The current file (943,346,336 bytes) carries the separate mm.pool.k / mm.pool.v tensors and the baked patch row. Re-download if you have the old one.

Conversion

• Source: allenai/Molmo2-8B, revision e28fa28597e5 (frozen 2026-01-23)
• Converter: llama.cpp HF-to-GGUF with Mu2's Molmo2 vision converter at commit c3d25ece9
• Text architecture: qwen3 (rope freq base 1,000,000 read from the source config, verified against Ai2's published config.json)
• mmproj: clip, 315 tensors (SigLIP ViT, pooling cross-attention, SwiGLU projector), separate k/v pooling tensors and the baked mm.image_patch_add row, projection_dim 4096
• Chat template embedded from the source repository

Verification (2026-09-28)

Measured on Mu2's CPU-only OG Rig (Intel i3-2100, no GPU), rebuilt projector:

• Text: verified. Through llama-server's completion endpoint, the prompt The capital of France is returns Paris. The capital of France is Paris. The capital of France is Paris. Tokenizer metadata, rope base, head counts and tensor types checked against Ai2's published config and found correct.

• Vision encode: verified. The image encoding graph runs to completion, 493,673 ms in one run and 502,118 ms in another, for a single 378x378 view. (An earlier card revision cited about 1.3 s; that figure did not come from this hardware and has been removed.)

• Vision generation: KNOWN ISSUE, unresolved. Given an image plus Describe this image in one sentence., this model emits The image<im_patch><frame_start> and then whitespace until the token budget is exhausted, with a 90-token budget as well as a 40-token one. It does not describe the image.

  What has been ruled out by direct comparison against the working Molmo2-4B release: the chat template is byte-identical (3,169 characters, zero diff), the tokenizer metadata is identical, the projector metadata is identical apart from the expected projection_dim, and every tensor is present with the expected shape and type. The 4B, running the identical tooling, describes the same image correctly.

  This is a fault in the interface between this model's image embeddings and its text model, not a weights or a projector-format problem, and it needs a per-layer comparison against a transformers reference to pin down.

• This conversion was produced with Mu2's experimental llama.cpp fork (upstream llama.cpp does not yet support the Molmo2 vision graph).

Usage

# Text-only chat
llama-server -m Molmo2-8B-text-Q4_K_M.gguf --jinja -c 8192

# Vision (Mu2 experimental mtmd tooling; see the generation issue above)
llama-mtmd-cli -m Molmo2-8B-text-Q4_K_M.gguf \
  --mmproj Molmo2-8B-mmproj-F16.gguf \
  --image image.jpg -p "Describe this image."

Credit

• Model: Allen Institute for AI (Ai2), Apache 2.0
• Conversion: Mu2 Solutions, promoting truly open-source AI, one GGUF at a time