Downloads · 30 days
141
21% of all-time downloads
curvedinf/Qwen3.8-27B-DFlash2-GPTQ-INT8-W8A8-GS128
Qwen3.8-27B-DFlash2-GPTQ-INT8-W8A8-GS128 is a text generation model from curvedinf. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This DFlash2 draft is designed to run as the speculative-decoding companion to the matching Qwen3.8-27B target model. It is not a standalone model.
Downloads · 30 days
141
21% of all-time downloads
All-time downloads
676
Public
Parameters
1.9B
2.3 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors2.3 GB · 99%
How the weights are stored.
I321.6B · 83%
From the Hugging Face model README
This DFlash2 draft is designed to run as the speculative-decoding companion to the matching Qwen3.8-27B target model. It is not a standalone model.
GPTQ INT8 quantization of
z-lab/Qwen3.8-27B-DFlash2,
the DFlash 2 draft model for Qwen/Qwen3.8-27B.
This is not a standalone language model. It is a five-layer block-diffusion
drafter used by a speculative-decoding server alongside the matching target
model linked above. The checkpoint is intended for the custom W8A8 serving
path in curvedinf/int8-vllm.
| Parameter | Value |
|---|---|
| Format | GPTQ |
| Weight precision | INT8 |
| Intended serving mode | W8A8 |
| Group size | 128 (GS128) |
| Symmetric | true |
| desc_act | false |
| Draft layers | 5 |
| Quantized matrices | 35 |
| Quantizer | GPTQModel 7.3.4 |
All 35 packed GPTQ matrices were checked against their scale and g_idx
tensors. Every matrix uses contiguous groups of exactly 128 weights.
Use this draft with the linked target model and a DFlash2-capable build of the custom vLLM fork:
VLLM_ROCM_USE_AITER=1 \
VLLM_ROCM_USE_AITER_UNIFIED_ATTENTION=1 \
VLLM_GFX908_INT8_LM_HEAD=1 \
VLLM_GFX908_ACT_QUANT=round \
VLLM_DISABLED_KERNELS=TritonW8A16LinearKernel \
vllm serve curvedinf/Qwen3.8-27B-GPTQ-INT8-W8A8-GS128 \
--tensor-parallel-size 4 \
--max-num-seqs 8 \
--dtype half \
--kv-cache-dtype int8_per_token_head \
--mamba-ssm-cache-dtype float32 \
--speculative-config '{
"method": "dflash",
"model": "curvedinf/Qwen3.8-27B-DFlash2-GPTQ-INT8-W8A8-GS128",
"num_speculative_tokens": 15,
"kv_cache_dtype": "int8_per_token_head"
}'
DFlash 2 drafts candidate tokens; the target model verifies them. Speculative
decoding therefore preserves the target model's output distribution. The
top-level KV flag configures the target and the nested field configures this
draft; both use INT8 per-token-head KV. This is one fixed TP4/C8 contract:
AITER W8A8 INT8 GEMMs at every M, AITER unified attention, vLLM custom
all-reduce, fp32 Mamba/GDN state, and the two linked GS128 checkpoints.
--dtype half is the residual dtype; the AITER GEMM path dynamically
quantizes its inputs to INT8.
z-lab/Qwen3.8-27B-DFlash2z-lab/dflashcurvedinf/int8-vllm