Qwopus3.8-27B-Flash-TQ3_4S

Qwopus3.8-27B-Flash-TQ3_4S

TurboQuant TQ3_4S GGUF build of Jackrong/Qwopus3.8-27B-Flash, a speed-optimized fine-tune of Qwen/Qwen3.8-27B designed for fast agent workloads. The build retains the model's bundled NextN/MTP draft head for self-speculative decoding, and ships the upstream vision projector (mmproj) for image/video input.

Files

The file retains the full MTP/NextN draft head (blk.64.nextn.*) — quantized directly from the official BF16 weights, no importance-matrix dependence.

Required Runtime

The TQ3_4S tensor type is custom — use turbo-tan/llama.cpp-tq3. Stock llama.cpp builds cannot load TQ3_4S files.

Recommended Runtime (validated)

RTX 3090 24 GB — the dense 27B architecture fits fully on-GPU:

llama-server -m Qwopus3.8-27B-Flash-TQ3_4S.gguf \
  -ngl 99 -c 32768 -fa on -np 1 \
  --reasoning-format deepseek --reasoning-budget 4096 --jinja

Measured decode: ~48 tok/s (single stream, 3090). Add --spec-type draft-mtp for self-speculative decoding with the bundled draft head (~60 tok/s on code workloads).

Benchmark Results (RTX 3090)

Evaluated at reasoning budget 4096, temperature 0.0, seed 42, official scorers:

The HumanEval-vs-MBPP gap reflects the Flash tuning objective: terse, agent-oriented completions over exploratory algorithmic coding.

Comparison vs Qwen3.8-27B-TQ3_4S (same tensor format, RTX 3090)

Same base architecture and quant format; the Flash build is the agent-speed fine-tune of it. Protocols differ per each card — labeled below, compare within rows only where marked equal.

How to read this: the base model is the stronger general coder (HumanEval/MBPP at its own budget-256 protocol are far ahead — note the budgets differ, but the gap is much larger than any budget effect we've measured). Flash gives that up for its design goal: fewer tokens per task, faster task completion in agent loops, and slightly better unit-tested coding (Hard86). Pick the base card for raw capability; pick Flash for agents where completion latency dominates.

MTP Speculative Decoding

Measured draft acceptance with --spec-type draft-mtp (self-speculative, no second model, RTX 3090):

Weighted across the 4-prompt probe: 59.0% (2150/3646 drafted tokens accepted). With MTP enabled, measured decode rises from ~48 to ~60 tok/s on code workloads. The draft head is quantized directly from BF16 (the TQ3_4S path needs no importance matrix), so these figures reflect the head as-shipped.

Quick Smoke Test

curl -s http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages":[{"role":"user","content":"Write an iterative Fibonacci function in Python."}],
  "max_tokens":2048, "temperature":0
}'

Expect a reasoning_content block followed by clean code — finish_reason stop.

Quantization Details

Direct quantization from the official BF16 GGUF (llama-quantize ... TQ3_4S). All tensors including the MTP/NextN head converted in a single pass; no importance matrix required for this format.

License

Apache 2.0 (inherits base model license).