Qwen3.5-9B-Q4KM-dequant-bf16-vllm

Qwen3.5-9B Q4_K_M GGUF dequantized BF16 for vLLM

Private Exolabs checkpoint converted from unsloth/Qwen3.5-9B-GGUF file Qwen3.5-9B-Q4_K_M.gguf.

The upstream Qwen/Qwen3.5-9B repository was used for config, tokenizer, chat template, and generation config only. Weight tensors come from the Q4_K_M GGUF and were dequantized in fp32, transformed back to HF/vLLM layout where llama.cpp stores Qwen3.5 Gated DeltaNet tensors differently, then cast once to BF16.

Validated on vLLM 0.23.0 with:

VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve . \
  --served-model-name qwen35-9b-gguf-q4km-bf16 \
  --dtype bfloat16 \
  --max-model-len 4096 \
  --attention-backend triton_attn \
  --gdn-prefill-backend triton \
  --trust-remote-code \
  --language-model-only

Smoke tests:

• The capital of France is -> starts with Paris.
• Q: What is 2+2? A: -> starts with 4.