Qwen3.8-27B-EES-ForgeMix-DFlash2-uint2-v1.0

EES ForgeMix DFlash2 uint2 v1.0

This repository contains the companion speculative draft for EES ForgeMix v1.0. It is a TP2 SGLang/Humming uint2 preshard derived from incoai/Qwen3.8-27B-DFlash2, which in turn targets Qwen/Qwen3.8-27B.

This is not a standalone language model. It cannot generate authoritative responses by itself. A compatible Qwen3.8 target model verifies accepted draft tokens inside a speculative-decoding runtime.

This is a third-party quantized and repackaged derivative. It was not trained, developed, owned, or endorsed by the Qwen team, Alibaba Cloud, In.co, Z-Lab, SGLang, or NVIDIA. Consult the upstream Qwen and DFlash2 model cards for original training, capabilities, intended use, limitations, and notices.

Artifact

The full-precision root draft weight is deliberately omitted. The signature directory contains four safetensor files, checksum.json, and a READY marker. Do not rename, flatten, merge, or move files out of that directory.

Exact Pairing

Validated target:

ersensari/Qwen3.8-27B-EES-ForgeMix-v1.0

Validated runtime profile:

TP2
64,000 target context and max-total-tokens
6 speculative draft tokens
page size 1
Mamba full-memory ratio 0.50
concurrency 1, FCFS, seed 42
target KV FP8 E4M3
draft KV BF16
FlashInfer attention, draft attention, sampling, and Mamba
NCCL_P2P_DISABLE=1
--disable-custom-all-reduce

Use the exact Docker build and launch command in the target repository's model card. Mount this repository root as /models/draft and pass:

--speculative-algorithm DFLASH \
--speculative-draft-model-path /models/draft \
--speculative-draft-attention-backend flashinfer \
--speculative-draft-kv-cache-dtype bfloat16 \
--speculative-draft-model-quantization humming \
--speculative-draft-load-format presharded \
--speculative-num-draft-tokens 6

The runtime also requires:

export SGLANG_HUMMING_ONLINE_QUANT_CONFIG='{"weight_dtype":"uint2","group_size":64,"has_zero_point":true,"force_requant":true}'

Validated System

The release pair was tested on 2x NVIDIA RTX 5060 Ti 16 GB (SM120), PCIe PHB without P2P/NVLink. With EES ForgeMix v1.0, the fair 20-request seed-42 workload completed 20/20 at 93.37 output tok/s with mean accepted length 3.11.

The native block capacity is 8, but 8 draft tokens is not the release setting. It improved throughput to 96.65 tok/s while reducing HumanEval+ from 0.896 to 0.878. Six draft tokens is the fully evaluated recommendation.

Acceptance and speed depend on target weights, prompts, sampling, context, runtime kernels, and draft count. Pairing with another Qwen3.8 target is unvalidated even when loading succeeds.

Integrity

presharded/TP-2-sig-fd5299d4cf6c66c4/checksum.json is the serialization's canonical checksum manifest. Verify it before serving. SHA256SUMS at repository root covers the complete public package.

Limitations And Responsible Use

• The preshard is tied to TP2 and its recorded SGLang/Humming serialization.
• It is not a standard Transformers standalone checkpoint and does not include a root model.safetensors.
• Do not infer output quality from draft-only metrics. The target, runtime, and verification path determine final behavior.
• Runtime changes can alter observed greedy outputs; validate quality after changing draft count, page size, Mamba allocation, kernels, or sampling.
• The draft does not add a safety layer. Upstream and target model risks remain.
• Generated code and tool calls require review, sandboxing, least privilege, and application-specific safeguards.
• Operators remain responsible for privacy, access control, acceptable use, regulatory compliance, reliability, and deployment security.

The artifact and modifications are provided on an AS-IS basis without warranties or conditions of any kind. See LICENSE for the controlling Apache-2.0 terms and limitation of liability.

Attribution

• Upstream target: Qwen/Qwen3.8-27B
• Upstream draft: incoai/Qwen3.8-27B-DFlash2
• DFlash project: z-lab/dflash
• Serving engine: sgl-project/sglang

See NOTICE for modification and redistribution notices.