Qwen3.8-Flash-Next-Uncensored-W4A16-AutoRound-INT8PLE-MTP

Qwen3.8-Flash-Next-Uncensored — AutoRound W4A16 / INT8 PLE

This is a locally assembled and tested quantization of orcarouter/Qwen3.8-Flash-Next-Uncensored, which derives from Qwen/Qwen3.8-Flash-Next. The language-model routed experts were quantized directly from the OrcaRouter BF16 checkpoint with standard AutoRound; this is not a refusal edit applied to an already-quantized Intel checkpoint. The model also carries an independently stored INT8 row-wise PLE n-gram embedding and retains the vision and MTP paths.

Safety: The source model has had refusal behavior deliberately removed. It has no reliable built-in safety guardrails. Use only for lawful, controlled research or applications with appropriate external moderation and abuse prevention. Do not expose it to end users without safeguards.

Verified serving stack: QSA HiSparse. This maintained, Apache-2.0 SGLang source fork implements Qwen Sparse Attention with CPU KV offload and supports the INT8 row-wise PLE offload used here. The project's runtime guide, setup instructions, and service controller are the starting points for reproducing this deployment.

The fork's current main is commit c8dbe6b4 (the maintenance PR #7), which integrates upstream SGLang 35f3c96ff4. That upstream commit is a fixed integration point, not a claim about the newest upstream main. The deployment validated on 2026-10-04 ran source revision 897286b12a, whose runtime code is identical to the reviewed test revision 2fe0731e03 — the two differ only in documentation files. An independent diff from 897286b12a to the merged main commit c8dbe6b4 shows only PROVENANCE.json and VALIDATION.md, so main carries the same runtime code as well; c8dbe6b4 itself was not executed here. The quantitative results further down were produced on the older pinned revision ee8fe158 and keep that attribution; the maintenance record is in VALIDATION.md.

Provenance and quantization

The independent PLE sidecar is a deliberate mixed-provenance serving choice. The OrcaRouter checkpoint's modified PLE projection weights are retained; the large n-gram embedding is supplied by the row-wise INT8 sidecar. Selected source PLE tensors were checked during assembly, but this card does not claim an exhaustive BF16-vs-INT8 numerical equivalence proof for the table. The assembly used copies rather than hard links, so this release does not depend on the local donor files.

Latest maintenance and validation (2026-10-04)

The serving fork was updated and re-validated on 2026-10-04. The maintenance merge fixed FP8 KV descales on the cached-prefix and reference read paths, migrated the cache lifecycle and factory wiring to upstream's new calls, and adopted upstream's parallel interfaces (tp_rank and the TP CPU group). The checkpoint, its quantization and the serving configuration were not changed.

These are functional checks of one configuration. They are not a full-model bitwise or determinism proof, and they are not a performance improvement claim. Full protocol, environment versions and configuration are in BENCHMARK.md.

Related quantizations

bidhata/Qwen3.8-Flash-Next-AutoRound-Uncensored is a close relative, but its card describes transferring an uncensoring edit onto a pre-quantized Intel/Saren AutoRound hybrid and using an FP8 PLE table. Saren/Qwen3.8-Flash-Next-W4A16-AutoRound-hybrid is an AutoRound-based serving hybrid of the standard, non-uncensored Qwen model. This release differs in the direct OrcaRouter-BF16 quantization path and its INT8 row-wise PLE. It is not presented as the first or only Uncensored/AutoRound quantization.

Evaluation

The full frozen EvalPlus code-generation run used one greedy completion per task, max_tokens=1536, thinking disabled, and EvalPlus 0.3.1. Generated code was scored only in an offline, unprivileged container. All 542 API requests completed; two outputs reached the generation limit. The serial scorer produced 542 valid rows and zero internal scorer errors.

The stricter plus score is the primary public code-generation result. Scores on these established datasets can be affected by training-data overlap, so they are not proof of generalization. Detailed aggregate counts and the frozen protocol are in benchmark/evalplus.json and BENCHMARK.md.

The DSH coding-agent run passed 24/24 project-owned validation tasks (0 failures, 0 errors), using 150 local model calls and 131 tool calls. Median end-to-end task time was 22.246 seconds; the sum of task times was 589.304 seconds. This measures a small, non-blind tool-using workflow rather than a public general-agent score. Task texts, workspaces and transcripts are not redistributed here; the recovery gap in the local aggregator is excluded from the reported task times. See BENCHMARK.md and the aggregate JSON.

Earlier serving checks on the same local checkpoint and qsa-hisparse@ee8fe158d64186b47236b007a299696030c372e8 completed 704/704 requests in a 10-minute B8 soak with zero observed request errors. A separate fixed long-decode test measured 80.493 output tokens/s for one request after warmup. These figures describe specific local serving workloads, not an architecture-wide speed claim or a controlled A/B against the BF16 source.

Serving

The verified runtime is the QSA HiSparse SGLang fork, tensor-parallel on two GPUs with 48 GiB memory each — modified cards, not stock 24 GiB parts — and CPU-backed PLE offload. The measurements in BENCHMARK.md were produced on the pinned revision ee8fe158; the 2026-10-04 maintenance and validation above were produced on the reviewed revision 2fe0731e03 and deployed as 897286b12a, whose runtime code is identical. Other SGLang, Transformers, vLLM and llama.cpp loading paths are not verified for this mixed format. The essential runtime options were:

python -m sglang.launch_server \
  --model-path /path/to/model \
  --served-model-name qwen3.8-flash-next-uncensored-w4a16-int8ple \
  --model-impl sglang --tp-size 2 \
  --dtype bfloat16 --ple-offload-embedding \
  --json-model-override-args '{"text_config":{"ple_embedding_dtype":"int8_row"}}' \
  --kv-cache-dtype fp8_e4m3 --linear-attn-backend triton \
  --reasoning-parser qwen3 --tool-call-parser qwen3_coder

The short command above is a loading example, not the measured full profile: the measured deployments additionally set the cache, memory, Mamba, NUMA and CUDA-graph options listed in BENCHMARK.md. The INT8 PLE sidecar is required; do not omit model-ple-rowint8.safetensors. The current serving profile does not enable speculative decoding, so the retained MTP weights are not exercised as an acceleration path here.

License and limitations

The actual LICENSE distributed with the official Qwen base and this model is the Qwen Community License 1.0. It permits publication of derivatives subject to its conditions, including preserving its notice. In particular, commercial Model-as-a-Service or AI Work Assistant use by a licensee may require a separate Qwen license; read the full included LICENSE before using the model. The OrcaRouter source model card currently labels itself Apache-2.0, which conflicts with the official Qwen base repository and its license file. This release does not present the Qwen-origin weights as Apache-only.

Only a narrow set of coding, agent and serving workloads has been tested here. The BF16 source was not run locally as a matched benchmark arm, so differences cannot be attributed solely to quantization. The 2026-10-04 window adds a small set of reviewed functional checks — including one native image-read task and one proof task for a known problem — but there is still no system-level capability, long-context, security or broad safety benchmark for this release, and the image-read check is not a multimodal capability evaluation. The maintenance window is also not an all-green upstream CI run: repository formatting debt and hardware-label gates are still open, which does not change the locally reported results, and no wheel packaging or AMD validation is claimed.