Qwen3.8-27B-W4A8

Qwen3.8-27B W4A8 (for ComfyUI)

W4A8 quantized version of Qwen3.5/Qwen3.8-27B (qwen3_5 architecture) as a single safetensors file, loadable directly by ComfyUI. Weights are compressed from 27.9GB (BF16) to 16.7GB.

Model Overview

• Base model: Qwen3.8-27B (Apache-2.0)
• Quantization format: asym_w4a8_int8 (4-bit weights, int8 activations)
• Grouping: weight group_size = 16, convrot_groupsize = 256
• Quantization algorithm: comfy-kitchen AsymW4A8Int8Layout
  • ConvRot Hadamard rotation (256-blocks) followed by per-group quantization
  • Per-group 16-level Lloyd-Max codebook + fp8 group scales + fp32 channel scales
• File size: qwen3.8-27b-w4a8.safetensors ≈ 16.7GB

Conversion Scope

ComfyUI Usage

1. Put qwen3.8-27b-w4a8.safetensors into ComfyUI/models/text_encoders/.
2. Select the file in a CLIP Loader. ComfyUI auto-detects it as QWEN35_27B (detect_te_model) and enables mixed-precision layers (llama_detect → llama_quantization_metadata).
3. Quantized layers run through the AsymW4A8Int8Layout CUDA/Triton kernels.

Accuracy

• Dequantizing a single layer (in_proj_qkv) reproduces comfy-kitchen's quantization of the original BF16 weight bit-for-bit (relative error 0.00000, checks passed).
• Typical W4A8 forward error vs full-precision BF16 is ~1% (varies by layer and data distribution).
• It is recommended to run your own quality evaluation before deploying.

Hardware Requirements

• NVIDIA GPU (Ampere or newer, CUDA 13.0+) recommended for the optimized comfy-kitchen CUDA backend; CPU / Triton fall back gracefully.
• Conversion measured on RTX 4060 (8GB VRAM) from 52GB BF16 shards: ~4 minutes.

Conversion Tool

Quantized with convert_qwen35_w4a8.py (comfy-kitchen quantization backend), shipped alongside the ComfyUI repo:

python convert_qwen35_w4a8.py \
    --input-dir ./qwen38_27b \
    --output ./qwen3.8-27b-w4a8.safetensors \
    --verify

License & Disclaimer

The base model is Apache-2.0; this quantized model is released under the same license. This is a community quantization and differs from the original weights in accuracy. Use at your own discretion; please comply with the base model's terms of use and applicable laws in your jurisdiction.