Molmo2-O-7B-GGUF

Molmo2-O-7B GGUF

GGUF conversion of allenai/Molmo2-O-7B, published by Mu2 Solutions. No GGUF of Molmo2-O-7B was publicly available when this was built (checked 2026-09-28).

Molmo2-O-7B is Allen Institute for AI's fully open multimodal model (Apache 2.0): a SigLIP vision encoder plus a pooling cross-attention projector over a fully open 7B text backbone (olmo2 architecture, per-head QK-norm).

License

Apache 2.0, same as the source model. No additional restrictions.

Files

Load the text quant together with the mmproj to get a working vision model.

Vision requires Mu2's experimental llama.cpp fork. Upstream llama.cpp does not yet support the Molmo2 vision graph, so this mmproj will not load on a stock build. See the 8B repo for the full explanation.

Conversion

• Source: allenai/Molmo2-O-7B, revision 784410650d12 (frozen 2026-01-23)
• Converter: llama.cpp HF-to-GGUF with Mu2's Molmo2 vision converter at commit c3d25ece9
• Text architecture: olmo2
• mmproj: clip, 315 tensors (SigLIP ViT, pooling cross-attention, SwiGLU projector), with separate mm.pool.k / mm.pool.v tensors and the baked mm.image_patch_add row
• Chat template embedded from the source repository

Verification (2026-09-28)

Measured on Mu2's CPU-only OG Rig (Intel i3-2100, no GPU):

• Vision encode: completed in 491,094 ms for one 378x378 view.

• Vision generation: asked Describe this image in one sentence. on a real image, the model answered:

  The image displays a partially visible logo or advertisement with large, bold text in blue and red colors, featuring fragmented words like "RE-V," "BUSINESS," and "STRA," along

  The image is a corporate wordmark on a white background with navy and dark red lettering. The text, the colours and the layout are all correct, so this is a genuine reading of the image rather than a plausible guess.

Command used:

llama-mtmd-cli -m Molmo2-O-7B-text-Q4_K_M.gguf \
  --mmproj Molmo2-O-7B-mmproj-F16.gguf \
  --image image.jpg -c 4096 -t 4 -n 40 --temp 0 \
  -p "Describe this image in one sentence."

Requirements and known issues

• Needs a llama.cpp build with Molmo2 mtmd support. Upstream llama.cpp does not support the Molmo2 vision graph yet; this release was produced and verified with Mu2's fork.
• CPU-only encoding is slow. The vision tower is F16 and this machine has no F16C, so a single view took about eight minutes. A GPU or a modern CPU changes this by orders of magnitude.
• Frame markers can leak into the text. This run appended two literal <frame_end> tokens after the sentence. Cosmetic; the description itself is intact.
• Token-type warning at load. llama.cpp may warn that the </s> token is not control-type and overrides it. The same warning appears on our Molmo2-4B and Molmo2-8B files.

Usage

llama-mtmd-cli -m Molmo2-O-7B-text-Q4_K_M.gguf \
  --mmproj Molmo2-O-7B-mmproj-F16.gguf \
  --image photo.jpg -p "Describe this image."