Qwen3.8-27B-TURBO-Fable-Cold-Fusion-NVFP4-NInfer

Qwen3.8-27B TURBO Fable Cold-Fusion — NVFP4 NInfer artifact

A NInfer v3 artifact of DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU in the same physical layout as the official neroued/Qwen3.8-27B-nvfp4-NInfer: NVFP4 (W4A4) MLP projections in layers 0–55, row-scaled FP8 everywhere else in the text stack, groupwise-int Vision and MTP weights, plus the indexed proposal head. Same kernels and the same performance class as the official artifact; only the weights and the chat template differ.

Nothing was requantized from scratch. The NVFP4 rows are imported unchanged from Solstice-AI's calibrated compressed-tensors quant of this base; the FP8 rows are produced from DavidAU's BF16 weights by NInfer's own converter (fp8_row_maxabs, dynamic per-token activations at runtime, no calibration involved). The conversion recipe is included in this repository (davidau_qwen3_8_27b_nvfp4.py) and is reusable for other Qwen3.8-27B finetunes that have a compressed-tensors NVFP4 quant.

The model is DavidAU's tune: see the base card for what "TURBO", "Cold Fusion", "Fable Fusion 711" and "Heretic" mean, and for the recommended samplers (temperature 0.6–0.7, repetition penalty off). The tune's own chat_template.jinja is embedded, so reasoning_effort (xhigh default, medium, low) and enable_thinking work through NInfer's request fields.

Artifact

Verify a downloaded file with:

printf '%s  %s\n' \
  '94af134b33da1860bb70b3ae34671d81465e42731b8196205bee8d97433cbac6' \
  'qwen3_8_27b_turbo_fcf_nvfp4.ninfer' | sha256sum --check

Requirements

• NInfer at revision 9e163eee (2026-09-18) or later, built from source — the revision this artifact was converted and served with;
• 64-bit Linux (WSL2 Ubuntu 24.04 works);
• NVIDIA GeForce RTX 5090 (sm_120a);
• CUDA Toolkit 13.1 or newer.

Download and start a server

hf download koldfrontier/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-NVFP4-NInfer \
  qwen3_8_27b_turbo_fcf_nvfp4.ninfer --local-dir models

./build/apps/ninfer-serve models/qwen3_8_27b_turbo_fcf_nvfp4.ninfer \
  --max-context 151552 --kv-capacity auto \
  --max-concurrency 1 --kv-dtype int8 \
  --spec mtp --draft-tokens 3 --lm-head-draft \
  --preserve-thinking --vision

Without --vision, the full native context fits: --max-context 262144 --kv-capacity auto --kv-dtype int8 loaded with 19.7 GiB of weights, a 9.30 GiB runtime and 1.25 GiB left (nvidia-smi: 30,636 MiB used of 32,607) on this artifact. A request:

curl http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "qwen3.8-27b-turbo-fcf",
  "messages": [{"role": "user", "content": "Reply with one short sentence."}],
  "max_tokens": 64, "temperature": 0.6, "reasoning_effort": "medium"
}'

Measurements

All numbers below were taken on one RTX 5090 (32 GB, driver 610.62) under WSL2 Ubuntu 24.04, NInfer 9e163eee, --kv-dtype int8, MTP with 3 draft tokens, --vision loaded, --max-context 151552, single request. They are single runs on a handful of prompts, not a benchmark suite; the publisher's protocol for the official artifact is documented in docs/performance.

Perplexity (NInfer's fixed corpus, same protocol for both artifacts)

ninfer-perplexity --corpus eval/corpora/perplexity-1m/manifest.json --kv-dtype int8 (full corpus, 16 streams, context 4,096 / stride 2,048, 1,044,557 scored tokens).

This compares two different models under one conversion layout; it shows the conversion produced a sound model, not that either model is "better". No comparison against the BF16 base is available (NInfer does not run BF16 checkpoints).

Throughput (single request)

The code row, the image probe and the 262K capacity check were re-run on the released file; the prose, thinking and 147K rows were measured on the pre-release build of the same conversion (byte-identical text weights; it only additionally carried the DFlash2 objects).

Decode speed depends on how often the MTP head's drafts are accepted; on this tune's output the acceptance rate is higher than on stock Qwen3.8-27B for code and reasoning, which is where its decode advantage comes from. The prose row is at parity. Treat the code row as indicative (one sample each, different completion lengths).

Vision: an image probe (shapes, colours and embedded text) was described correctly with --vision. Video was not tested.

How it was built

python -m tools.convert \
  --model DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU \
  --recipe davidau_qwen3_8_27b_nvfp4.py \
  --source quantized=Solstice-AI/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-NVFP4 \
  --components text,vision,mtp \
  --resource chat_template.jinja=<base>/chat_template.jinja \
  --resource tokenizer.json=<Qwen/Qwen3.8-27B>/tokenizer.json \
  --resource tokenizer_config.json=<Qwen/Qwen3.8-27B>/tokenizer_config.json \
  --proposal --name qwen3.8-27b-turbo-fcf --device cpu \
  --out qwen3_8_27b_turbo_fcf_nvfp4.ninfer

The recipe mirrors the official qwen3_8_27b_nvfp4 recipe. Where the official recipe imports FP8 rows from unsloth's mixed checkpoint, this one generates them from the BF16 base with fp8_row_maxabs, because the community NVFP4 quants of this base are all-Linear NVFP4 and carry no FP8 tensors. The result was checked object-for-object against the official artifact with tools.artifact.inspect: identical tensor shapes, formats, layouts, bindings and activation uses for every object present in both (this artifact omits only the DFlash2 objects). CPU conversion took about 10 minutes.

Limits

• No DFlash2. The z-lab/Qwen3.8-27B-DFlash2 draft was trained on stock Qwen3.8-27B; tested against this tune it accepted 3–5% of drafts and decode fell to ~70 tok/s, so the component was left out. Use --spec mtp (the MTP head is DavidAU's, trained with the model).
• NInfer runs on one RTX 5090 with a startup-fixed capacity of 1–8 active requests; no multi-GPU, offload, preemption or distributed serving. NInfer does not execute tool calls.
• The tune's "TURBO" reasoning-length reduction was not measured here; thinking length varied 1.5K–6K tokens across single samples at temperature 0.6.
• This is an uncensored ("Heretic") model. Use it responsibly; the base card describes its training and intent.

Provenance

artifact-manifest.json summarizes the artifact identity and object inventory in the same shape as the official artifacts' manifests; SHA256SUMS covers the artifact file; perplexity-report.json is ninfer-perplexity's full-precision report for the table above (per-window, per-stream and per-domain NLL/PPL).

License

This artifact is distributed under the Apache License 2.0. The base model repository (DavidAU), the quantized source repository (Solstice-AI), the tokenizer source (Qwen/Qwen3.8-27B) and NInfer are all Apache-2.0. Users remain responsible for complying with the license and applicable laws. Credit for the model belongs to DavidAU and the contributors named on the base card; credit for the NVFP4 calibration to Solstice-AI; credit for the engine and the conversion tooling to Neroued.