MANGO1.5-Qwen3.5-9B-GGUF

MANGO1.5-Qwen3.5-9B-GGUF

GGUF quantizations of CMKL/MANGO1.5-Qwen3.5-9B, CMKL University's dense 9B bilingual (Thai/English) fine-tune of Qwen/Qwen3.5-9B, for use with llama.cpp and any runtime built on it (Ollama, LM Studio, koboldcpp, text-generation-webui, etc.).

See the base model card for full details on training data, evaluation results, and intended use. This repo only covers the GGUF conversion/quantization itself.

About this conversion

• Source: merged full-precision (bf16) checkpoint, converted with a llama.cpp build carrying native support for the Qwen3.5 hybrid linear-attention (Gated DeltaNet) architecture (general.architecture = qwen35).
• Scope: text-only. The base Qwen3.5 architecture is natively multimodal (it ships a vision tower), but this fine-tune was trained text-only and the vision tower is not included in these GGUFs — there is no accompanying mmproj file, and these files will not do image/video input.
• Speculative decoding (MTP): the source checkpoint includes Qwen3.5's Multi-Token-Prediction (NextN) draft layer, and it is included here (qwen35.nextn_predict_layers = 1, 33 blocks total: 32 main + 1 MTP). Whether your runtime actually uses it for self-speculative decoding depends on that runtime's support for qwen35's NextN layer — check your llama.cpp build's release notes if in doubt.
• Vocab / chat template: the Qwen3.5 BPE tokenizer and Jinja chat template are embedded in the GGUF metadata, including the bilingual system-prompt default and <think>/<tool_call> hybrid-reasoning template from the base model.

Files

All files were produced from the same BF16 base with llama-quantize; no calibration/importance-matrix data was used for any of them (including IQ4_XS), so IQ4_XS in particular may underperform its usual reputation relative to the K-quants.

Which one should I use?

• Not sure / general use: Q4_K_M.
• Have 12GB+ VRAM and want better quality: Q5_K_M or Q6_K.
• Want maximum fidelity / plan to requantize yourself: BF16.
• Squeezed for space (e.g. 6-8GB VRAM): Q4_K_S, then IQ4_XS if you still need to shave more off.

How to use

llama.cpp

# CLI, one-shot
llama-cli -m MANGO1.5-Qwen3.5-9B-Q4_K_M.gguf -ngl 99 -p "Hello, who are you?"

# OpenAI-compatible server
llama-server -m MANGO1.5-Qwen3.5-9B-Q4_K_M.gguf -ngl 99 --port 8080

-ngl 99 offloads all layers to GPU; drop it (or lower the number) for CPU-only or partial offload. The chat template is embedded, so llama-cli's interactive/conversation mode and llama-server's /v1/chat/completions endpoint will apply it automatically — no need to hand-format the bilingual system prompt from the base model card, though you're free to override it per-request.

Ollama

Ollama is llama.cpp-based, so it's subject to the same architecture-support requirement below (recent Ollama build needed for qwen35).

Directly from this Hub repo (once uploaded — see note below), no local download step needed:

ollama run hf.co/CMKL/MANGO1.5-Qwen3.5-9B-GGUF:Q4_K_M

Swap Q4_K_M for any other quant tag from the Files table above.

From a GGUF file you already have locally, via a Modelfile:

cat <<'EOF' > Modelfile
FROM ./MANGO1.5-Qwen3.5-9B-Q4_K_M.gguf
EOF

ollama create mango1.5 -f Modelfile
ollama run mango1.5

No TEMPLATE directive is needed in the Modelfile — Ollama reads the chat template embedded in the GGUF metadata (same one llama-cli/llama-server use) automatically.

Requires a recent llama.cpp / Ollama build

Qwen3.5's hybrid linear-attention (qwen35 architecture) support is recent. If your llama.cpp build (or the llama.cpp version bundled inside your Ollama install) predates it, these files will fail to load with an unknown-architecture error — update to a current build (e.g. ghcr.io/ggml-org/llama.cpp:full-cuda / server-cuda on Docker Hub / GHCR, or build from source; ollama --version / ollama pull the latest release for Ollama) before reporting a loading issue.

License

Inherits the Apache 2.0 license of the base model. See Qwen/Qwen3.5-9B for exact terms.

Credits

• Fine-tune: CMKL/MANGO1.5-Qwen3.5-9B — Atikan Payungthong, Dr. Akkarit Sangpetch (CMKL University)
• Base model: Qwen/Qwen3.5-9B
• Quantization tooling: llama.cpp