Qwen3.5-9B-Fable-5-Quad-Stock-Q2_K-GGUF

ZeroXClem/Qwen3.5-9B-Fable-5-Quad-Stock-Q2_K-GGUF

This model was converted to GGUF format from ZeroXClem/Qwen3.5-9B-Fable-5-Quad-Stock using llama.cpp (master) on Modal. Refer to the original model card for more details on the model.

Rebuilt 2026-07-03: the original conversion inherited a stale mtp_num_hidden_layers = 1 from the merge config (mergekit drops the mtp.* tensors of models that not all donors share, but copies the base config verbatim), so the GGUF declared a 33rd block with no tensors behind it and failed to load in every llama.cpp-based runtime (LM Studio included: missing tensor 'blk.32.attn_norm.weight'). Rebuilt from the corrected config — inference weights are unchanged.

Use with llama.cpp

Install llama.cpp through brew (works on Mac and Linux)

brew install llama.cpp

CLI:

llama-cli --hf-repo ZeroXClem/Qwen3.5-9B-Fable-5-Quad-Stock-Q2_K-GGUF --hf-file qwen3.5-9b-fable-5-quad-stock-q2_k.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo ZeroXClem/Qwen3.5-9B-Fable-5-Quad-Stock-Q2_K-GGUF --hf-file qwen3.5-9b-fable-5-quad-stock-q2_k.gguf -c 2048

🖼️ Image input (mmproj)

mmproj-qwen3.5-9b-fable-5-quad-stock-f16.gguf in this repo is the model's vision encoder + projector, extracted from the same merge. LM Studio and llama.cpp pair it with the text GGUF automatically when both sit in the model folder — download both files and the model accepts images.

llama-mtmd-cli --hf-repo ZeroXClem/Qwen3.5-9B-Fable-5-Quad-Stock-Q2_K-GGUF --hf-file qwen3.5-9b-fable-5-quad-stock-q2_k.gguf \
    --mmproj mmproj -p "Describe this image" --image photo.jpg