Qwopus3.6-27B-v2-NVFP4-MTP-GGUF

Qwopus3.6-27B-v2-NVFP4-MTP-GGUF

NVFP4 quantized GGUF of Jackrong/Qwopus3.6-27B-v2-MTP-GGUF (Q8_0), with Multi-Token Prediction heads preserved for speculative decoding.

About Qwopus3.6-27B-v2

Qwopus3.6-27B-v2 is a reasoning-enhanced dense language model built on Qwen3.6-27B by Jackrong, using multi-stage curriculum learning with Trace Inversion datasets. It achieves 87.43% on MMLU-Pro (vs 84.86% for base Qwen3.6-27B) and 75.25% on SWE-bench Verified. The MTP variant adds Multi-Token Prediction heads for ~1.66× inference speedup with no accuracy loss.

Original Creator: Jackrong | License: Apache-2.0

Quantization Details

This is a mixed NVFP4/Q4_K quantization, created using llama-quantize with --tensor-type-file:

• Source: Q8_0 GGUF (28 GB) → NVFP4 mixed (15 GB)
• BPW: 4.60 (down from 8.50)
• Size reduction: ~46% (56% of original size)
• MTP heads (nextn.eh_proj) preserved in Q4_K for speculative decoding

Performance

Benchmarked on NVIDIA RTX PRO 6000 Blackwell (98 GB VRAM, sm_120):

vs Q8_0: NVFP4 is 1.72× faster at baseline and 1.92× faster with MTP speculative decoding.

Usage

llama.cpp Server

llama-server \
  -m Qwopus3.6-27B-v2-NVFP4-MTP.gguf \
  --mmproj mmproj-F32.gguf \
  -ngl 99 -c 262144 \
  -b 1024 -ub 1024 \
  --split-mode none --main-gpu 0 \
  --cache-type-k f16 --cache-type-v f16 \
  --spec-type draft-mtp \
  --spec-draft-model Qwopus3.6-27B-v2-NVFP4-MTP.gguf \
  --spec-draft-n-max 4 \
  --spec-draft-p-min 0.0 \
  --parallel 1 --mlock --no-mmap

Reproduction

# Create tensor-type-file
cat > tensor_types.txt << 'EOF'
attn_qkv=nvfp4
attn_q=nvfp4
attn_k=nvfp4
attn_v=nvfp4
attn_output=nvfp4
ffn_down=nvfp4
ffn_gate=nvfp4
ffn_up=nvfp4
EOF

# Quantize
llama-quantize --allow-requantize \
  --tensor-type-file tensor_types.txt \
  Qwopus3.6-27B-v2-MTP-Q8_0.gguf \
  Qwopus3.6-27B-v2-NVFP4-MTP.gguf Q4_K

Hardware Requirements

• NVIDIA Blackwell GPU (RTX 5090, RTX PRO 6000, B200, etc.) required for NVFP4 tensor core acceleration
• ~16 GB VRAM minimum for model weights
• Additional VRAM for KV cache (varies by context length)

Acknowledgments

• Jackrong — original Qwopus3.6-27B-v2 model and MTP GGUF conversion
• Alibaba Cloud / DAMO Academy — Qwen3.6-27B base model
• Unsloth — training framework and original GGUF conversions
• ggml-org/llama.cpp — inference engine and NVFP4 quantization support

License

Apache-2.0 (inherited from original model)