Swift-Qwen3.8-27B-NVFP4

Swift-Qwen3.8-27B-NVFP4

An NVFP4 / FP8 quantization of ukisai/Swift-Qwen3.8-27b, built to run on SGLang with the DFlash2 speculative drafter and FlashInfer attention. It was measured on an NVIDIA DGX Spark (GB10, 128 GB unified memory), where it serves as the author's daily coding model with a 524,288-token context.

Credit

All of the reasoning-efficiency work in this model is UkisAI's. Swift-Qwen3.8-27B is UkisAI's fine-tune of Qwen/Qwen3.8-27B. According to its model card, it uses 58.3% fewer thinking tokens at under 1% accuracy loss and runs 1.95x faster on several tasks. Please read and cite the original model card, and see ukisai.com for enterprise licensing.

The precision map comes from RadixArk/Qwen3.8-27B-NVFP4 (NVIDIA Model Optimizer). The draft model is maurienne-ai/Qwen3.8-27B-DFlash2-NVFP4-RTNcal, a calibrated NVFP4 build of incoai/Qwen3.8-27B-DFlash2. The serving recipe follows hasso5703/dgx-spark-qwen38.

What this release adds

Swift makes the model think less. This release makes each token cheaper and faster on Blackwell:

• NVFP4 weights, 20 GB instead of 54 GB of BF16. That leaves most of a 128 GB Spark for KV cache: a ~740,000-token pool at --mem-fraction-static 0.70, enough for one 524k-token session per box.
• Reasoning-trace calibration with local-Hessian weight scales (v3). RadixArk's recipe calibrates on news articles with max. This build calibrates the same precision map on text shaped like Swift's own output (decontaminated Codeforces and OpenR1-Math <think> traces plus UltraChat, in Swift's chat template) and chooses the NVFP4 weight scales with ModelOpt's local-Hessian search. Measured against BF16 Swift it is the closest of every build tried (see Fidelity to BF16).
• DFlash2 speculative decoding in SGLang: the 2B block-diffusion drafter proposes 16 tokens per step. Depth 16 measured 14% faster than 8 on thinking tasks.
• FlashInfer attention, FP8 KV cache, and a tested YaRN 524k configuration (below).

Comparison

Three configurations, each alone on its own identical DGX Spark. The last two use the same SGLang image, flags and draft model, so the only difference between them is the model.

<p align="center">_{The LiveCodeBench rows are a sample from LiveCodeBench v6 (<code>release_v6</code>, AtCoder abc387/abc388: 3 easy and 3 medium problems, 3 samples each, graded on the public tests). Swift v2 was run 6 times in total: 3 at the native window and 3 with YaRN 524k.}</p>

What the table says:

• Against plain Qwen3.8-27B on the same stack, Swift writes 27% fewer tokens on the LiveCodeBench sample (17.1k vs 23.4k) and 22% fewer on the thinking tasks. It finishes 28% and 26% sooner, with the same solve rate. That gain is UkisAI's fine-tune; the quantization keeps it.
• Against the previous llama.cpp setup, prefill is 2.2x faster, and thinking tasks finish 2.0x sooner. On the longer LiveCodeBench problems the gain is smaller (1.2x on the median pass). There this model wrote about 20% more tokens than the Q4_K_M GGUF, which uses most of its 1.4x faster decode. That gap is not llama.cpp's default min_p = 0.05: rerunning with min_p = 0.05 gave the same token counts.
• The LiveCodeBench sample is small. Read the solve counts as "no visible regression", not as a pass rate.

How it was measured: reasoning effort medium, temperature 1.0 / top-p 0.95 / top-k 20, streaming, times from the client, nothing else running on the box.

Fidelity to BF16

Measured against BF16 Swift-Qwen3.8-27B on the same SGLang build: token-level log-probabilities at every position of two corpora, 100 held-out reasoning/code/chat texts (172,884 tokens) and 60 of Swift's own regenerated answers (98,892 tokens), plus the task gates.

KL is computed over BF16's top-20 tokens, with any token missing from the NVFP4 top-20 charged the leftover probability mass; it overstates the true KL. BF16 decodes at about 4.4 tok/s on one Spark without a draft, this model at about 57 tok/s with the DFlash2 draft (same flags as below); both need about the same number of thinking tokens per task (4,591 vs 4,378 on the quality set). An MSE weight search (same corpus) landed between v2 and v3. Perplexity against BF16 is not used as the fidelity gate: the calibrated builds score lower perplexity than BF16 on the held-out texts (−5.8% MSE, −8.6% local-Hessian), which is a deviation from BF16 rather than an improvement.

Quantization

NVIDIA Model Optimizer 0.46.0, max calibration. The precision map is layer-for-layer identical to RadixArk/Qwen3.8-27B-NVFP4 (all 401 quantized layers compared):

Calibration (v2 and v3): 1,024 samples at up to 1,024 tokens (899k tokens), all in Swift's chat template:

• 384 Python <think> traces from open-r1/codeforces-cots (solutions_py_decontaminated)
• 256 traces from open-r1/OpenR1-Math-220k
• 192 UltraChat turns (thinking off)
• 192 cnn_dailymail articles

No LiveCodeBench data was used.

The held-out texts are drawn from the same three sources, well past the part of each stream used for calibration. The perplexity is high in absolute terms because the traces were written by another model; only the difference between the two builds matters. The MTP head (mtp.*) is not included, so use DFlash2 (below), not NEXTN/EAGLE. The script that produced this checkpoint is quantization/quantize_swift_nvfp4.py (--calib reasoning --calib-seq 1024 --algo local_hessian). Note for anyone reproducing: ModelOpt's MSE and local-Hessian searches only act on static NVFP4 weight blocks; with the dynamic blocks max uses, they silently calibrate nothing and export identical weights.

Serving (SGLang, one DGX Spark)

Pinned image lmsysorg/sglang:v0.5.19 (sha256:d6e7288627be8b02be88e4bba38e73f6d50e2826869f753c13a4c4385ab3eda9):

docker run --rm --gpus all --memory 100g --memory-swap 100g --shm-size 16g --network host --ipc=host \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  lmsysorg/sglang@sha256:d6e7288627be8b02be88e4bba38e73f6d50e2826869f753c13a4c4385ab3eda9 \
  python3 -m sglang.launch_server \
    --trust-remote-code --model-path jessedye90/Swift-Qwen3.8-27B-NVFP4 --tp-size 1 \
    --served-model-name qwen3.8-27b \
    --mem-fraction-static 0.70 \
    --attention-backend flashinfer --chunked-prefill-size 8192 \
    --disable-prefill-cuda-graph --cuda-graph-max-bs 8 --disable-flashinfer-autotune \
    --speculative-algorithm DFLASH \
    --speculative-draft-model-path maurienne-ai/Qwen3.8-27B-DFlash2-NVFP4-RTNcal \
    --speculative-draft-model-revision bd7a934213c47a9e7ef69eef36bb3325f47fd1f1 \
    --speculative-num-draft-tokens 16 --speculative-draft-model-quantization modelopt_fp4 \
    --mamba-radix-cache-strategy extra_buffer --mamba-ssm-dtype bfloat16 \
    --max-mamba-cache-size 96 --max-running-requests 8 \
    --enable-torch-compile --torch-compile-max-bs 4 --num-continuous-decode-steps 2 \
    --reasoning-parser qwen3 --tool-call-parser qwen3_coder \
    --host 0.0.0.0 --port 8000

The first boot takes 7–8 minutes (torch.compile and CUDA-graph capture). The server speaks OpenAI /v1/chat/completions, /v1/responses and Anthropic /v1/messages. Claude Code and Codex CLI both run against it with tool use. Image input was tested over the chat-completions API, including an image placed after 36k tokens of text.

Two DGX Sparks as one server (TP2)

Add --tp-size 2 --nnodes 2 --node-rank {0|1} --dist-init-addr <node-1 IP>:20000 on each Spark, pass the RDMA devices (--device=/dev/infiniband --ulimit memlock=-1 --cap-add=IPC_LOCK) and point NCCL at both ConnectX-7 RoCE links (NCCL_IB_HCA=rocep1s0f0,roceP2p1s0f0, NCCL_IB_GID_INDEX=3). Measured single session, same flags otherwise:

With TP2's pool, a 1M-token window (YaRN factor 4) also fits: a needle was found at 939,729 tokens (cold prefill 36 minutes). Factor 4 lowers agreement with BF16 on ordinary text (KL +11-16 % vs factor 2), so 524k remains the recommended window.

524k context (YaRN)

The native window is 262,144 tokens. To serve 524,288, copy the checkpoint, and separately the draft, to local directories. In both config.json files, set max_position_embeddings to 524288 and add "rope_type": "yarn", "factor": 2.0, "original_max_position_embeddings": 262144 to rope_parameters. The target keeps these under text_config; the draft keeps them at the root. The draft crashes at load if only the target is patched. Then add --context-length 524288 and -e SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1.

Measured with YaRN 524k:

• Held-out perplexity: 7.955 (8.066 native), so no short-context cost at factor 2.
• LiveCodeBench sample: 18/18. Quality set: 20/20.
• Needle: found at 50% depth of a 411,494-token prompt.
• Cold prefill of that prompt: 13.6 min. A follow-up on the cached prefix reached its first token in 1.7 s.
• Decode at 411k depth: 14 tok/s.

Raise your client's stream-idle timeout above the cold-prefill time.

Notes for GB10 / unified memory:

• --mem-fraction-static: 0.70 served 4–5 concurrent 125k-token prompts with at least 24.8 GB of host memory free. At 0.76 the scheduler crashed under 6 concurrent 125k prompts and exited with status 0, so use Restart=always in systemd, not on-failure. Above 0.80, GB10 can run out of unified memory and freeze, because SGLang does not account for 25–40 GB of transient allocations there.
• Reasoning effort: the Qwen3.8 chat template defaults to xhigh when a request doesn't set reasoning_effort. At xhigh this model thought about 2.5x longer, and failed our code-generation task on both llama.cpp and SGLang. To make medium the default, change reasoning_effort|default('xhigh') to reasoning_effort|default('medium') in a local copy of chat_template.jinja (and the chat_template field in tokenizer_config.json). Don't use --default-chat-template-kwargs '{"reasoning_effort": "medium"}'. In SGLang v0.5.19 that flag overrides every request's own reasoning_effort, so asking for low or xhigh silently has no effect.

License

This is a derivative of Swift-Qwen3.8-27B and is distributed under the same terms:

• The Swift Contribution is licensed by UkisAI under the Swift Open License v1.0 (LICENSE), including its Section 5 Commercial Use Limitation. Commercial use by an entity with US$1M or more in annual gross revenue needs a separate Swift Enterprise License from UkisAI (ukisai.com/contact).
• The Base Model, Qwen3.8-27B (Copyright 2026 Alibaba Cloud), is licensed under the Apache License 2.0 (LICENSE-APACHE-2.0).
• NOTICE carries UkisAI's attribution notice and the change notice for this quantization.

"UkisAI" and "Swift" are UkisAI's names and are used here only to describe where this model came from. This quantization is not made or endorsed by UkisAI, RadixArk, Alibaba Cloud or NVIDIA.

Copyright 2026 UkisAI. Swift Contribution licensed under the Swift Open
License v1.0 (https://huggingface.co/ukisai/Swift-Qwen3.8-27b/blob/main/LICENSE).
Derivative of Qwen3.8-27B, Copyright 2026 Alibaba Cloud, Apache License 2.0.