FabGemma-GGUF

<p align="center"> <img src="Fab.png" alt="FabGemma" width="600"> </p>

FabGemma — GGUF

GGUF quantizations of naazimsnh02/FabGemma, a merged fine-tune of Gemma 4 12B (unified), converted with llama.cpp.

These builds are multimodal: the unified vision projector (mmproj) is included, so you can run text and image + text. (Audio tensors exist in the unified projector but are experimental in llama.cpp.)

Files

Requires a recent llama.cpp build with Gemma 4 support (architecture gemma4 / projector gemma4uv). Older builds cannot load these.

Text

llama-cli -hf naazimsnh02/FabGemma-GGUF:Q4_K_M --jinja -p "Tell me a short fable about a fox."
# or local:
llama-cli -m FabGemma-Q4_K_M.gguf --jinja -p "Tell me a short fable about a fox."

Image + text (multimodal)

llama-mtmd-cli \
  -m FabGemma-Q4_K_M.gguf \
  --mmproj FabGemma-mmproj-F16.gguf \
  --image your_image.png \
  --jinja \
  -p "Describe this image in one sentence."

llama-server works too — pass both -m and --mmproj and use the OpenAI-style chat endpoint with image content parts.

Notes

• --jinja is required (the chat template is a Jinja template).
• Conversion: convert_hf_to_gguf.py (text) and convert_hf_to_gguf.py --mmproj (vision). FabGemma ships no preprocessor_config.json; the mmproj was built with Gemma 4 image normalization image_mean=[0,0,0], image_std=[1,1,1].