Qwen3.5-27B-GGUF

Qwen3.5-27B GGUF

Q4_K_M and IQ3_KT quantizations of the official Qwen/Qwen3.5-27B checkpoint, with model-matched BF16 and Q4_K_M multimodal projectors. Published by Advantech-EIOT; not an official Qwen quantization release.

Files and checksums

• Q4_K_M/Qwen3.5-27B-Q4_K_M.gguf: 16,547,399,232 bytes; SHA-256 989be6a8064cfd3e47a48a80f48799cc45e228262e729a79b85a9e2f888e00b2.
• IQ3_KT/Qwen3.5-27B-IQ3_KT.gguf: 11,003,203,136 bytes; SHA-256 0c4d21d7353898f4e3bfe1b86c9e511902c463076659694e9020f02d8a866458.
• mmproj/mmproj-Qwen3.5-27B-BF16.gguf: 931,145,984 bytes; SHA-256 2abfbdeed2794d147bfa9bfee7b7b1536755c2d79b5a7d3cd911b0104ad011de.
• mmproj/mmproj-Qwen3.5-27B-Q4_K_M.gguf: 522,293,120 bytes; SHA-256 e4dd117ca99e4de92d03a0e860736b36d4ba123d091f55836e9adaf991175c82.

Machine-readable tensor types, sizes and provenance: manifest.json.

Usage and compatibility

The tested runtime is a custom llama.cpp b8779-lineage build with Qwen3.5 VLM and IQ3_KT support. The upstream tutorial anchor is 75f3bc94e649616162981c322e8e6b88ca5491e8; this is not a claim that pristine upstream b8779 or arbitrary stock llama.cpp builds support these files. The custom runtime is not bundled here. Verify your runtime's architecture, projector and IQ3_KT tensor support before use.

With that compatible build, from the downloaded repository root:

GGML_CUDA_DISABLE_GRAPHS=1 /path/to/compatible/llama-server \
  -m Q4_K_M/Qwen3.5-27B-Q4_K_M.gguf \
  --mmproj mmproj/mmproj-Qwen3.5-27B-BF16.gguf \
  --host 127.0.0.1 --port 18779 \
  -c 2048 -b 256 -ub 128 -ngl 16 --fit off \
  -t 12 -tb 12 --parallel 1 --jinja --no-warmup

API: http://127.0.0.1:18779/v1/chat/completions. These are the tested partial-offload settings, not performance recommendations for every device. Respect occupied ports and available RAM/VRAM.

Select the IQ3_KT/ main model or the mmproj/ Q4_K_M projector for the other tested pairings. Never use a projector from another model size. Exact preset is IQ3_KT, not IQ3_K_T. Its preset/file_type 152 and tensor enum 154 are different identifiers.

The Q4_K_M projector uses native mixed precision (Q4_K, Q5_0, Q8_0, F16, F32) due to vision tensor shapes; not all weights are 4-bit. No language-model imatrix was applied to the projector.

Provenance

• Official source revision: fc05daec18b0a78c049392ed2e771dde82bdf654.
• Both main quantizations came from the same official BF16 conversion and the same accepted, model-specific VLM importance matrix. No third-party low-bit requantization.
• Q4_K_M: custom b8779 quantizer. IQ3_KT: ik_llama.cpp quantizer; inference remained on the custom b8779 runtime.
• Formal calibration: 288 samples; 497 entries; DAT SHA-256 ffc7d3c738ea19d9511e76934dc7a1581eb20fa0d0667241178b2087c4c6c143.

Functional validation and limitations

Both main quants were tested with both projectors: four pairings, 16 text/caption/OCR/spatial requests on the original custom runtime. All returned normally without token truncation. Text sanity and the book-title OCR case passed; captions described the main visible scene.

These are functional smoke checks, not an accuracy benchmark. Cases came from existing calibration data, so this is not independent generalization evidence. Spatial answers matched the dataset label, but the couch-edge image is visually ambiguous; agreement is not proof of correctness. No additional BF16-main quality benchmark or reference-logit comparison was performed.

Known retained issue: IQ3_KT + BF16 projector misread sign text in its caption as "50000 PASSENGERS"; main-scene description remained plausible.

License

Apache-2.0, inherited from the official source. See LICENSE and the linked base-model card for upstream information.