Qwopus3.8-27B-Flash-GPTQ-4bit

Qwopus3.8-27B-Flash GPTQ 4-bit (FOEM)

GPTQ 4-bit quantization of Jackrong/Qwopus3.8-27B-Flash using the FOEM (First-Order Error Matters) recipe, quantized with gptqmodel and validated end-to-end on vLLM (TP2, 262K context).

Attribution & Credits

This is a quantized derivative. All credit for the underlying model belongs to its original developers:

License: Apache-2.0 (inherited from the base and fine-tune).

Quantization Details

• Method: GPTQ 4-bit with FOEM error compensation (FOEM + GPTAQ hybrid)
• Group size: 32 (g32)
• Symmetric: yes · desc_act: false · lm_head: not quantized
• mse: 2.0 · damp_percent: 0.05 · true_sequential: yes
• FOEM: α=0.25, β=0.2 · act_group_aware: yes
• Tool: gptqmodel (FOEMConfig), calibration: C4 (128 samples)
• Format: gptq, int32-packed → vLLM Marlin/GPTQ kernel path

This reproduces the exact recipe used for the production btbtyler09/Qwen3.8-27B-GPTQ-4bit checkpoint so results are comparable.

Serving

Tested on vLLM 0.27.1 with the GDN FlashInfer decode backport, 2× RTX 5090 (TP2):

vllm serve deluxetiky/Qwopus3.8-27B-Flash-GPTQ-4bit \
  --tensor-parallel-size 2 --max-model-len 262144 --kv-cache-dtype fp8 \
  --gpu-memory-utilization 0.92 --reasoning-parser qwen3 \
  --gdn-decode-backend flashinfer --mamba-block-size 16 --enable-prefix-caching \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml

Measured A/B vs base (same quant, same stack)

Head-to-head against the stock Qwen3.8-27B (same FOEM g32 quant) on 2×5090 TP2:

Note: this A/B ran on two different physical 2×5090 boxes (same GPU/stack), so absolute concurrency figures should be re-validated on identical hardware; the completion-rate difference is cap-bound and box-independent.

Files

• 6 × model-*.safetensors (~20 GB total) + config, tokenizer, chat template.