Downloads · 30 days
483
6% of all-time downloads
d0xin/Swift-Qwen3.8-27B-Uncensored-NVFP4-LocalHessian-ActivationHeadroom-NInfer
Swift-Qwen3.8-27B-Uncensored-NVFP4-LocalHessian-ActivationHeadroom-NInfer is a image-text-to-text model from d0xin. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for ninfer. The card lists the license as other.
Related models: all models Swift-1.5-Qwen3.8-27B-Uncensored-BF16 · Swift-1.5-Qwen3.8-27B-Uncensored-FP8 · Swift-1.5-Qwen3.8-27B-Uncensored-FP8-NInfer · Swift-1.5-Qwen3.8-Flash-Next-NVFP4-FP8PLE · Swift-1.5-Qwen3.8-Fla…
Downloads · 30 days
483
6% of all-time downloads
All-time downloads
8.7K
Public
Repo size
23.7 GB
Likes
2
Public
Click a slice to open those files.
.ninfer23.7 GB · 100%
From the Hugging Face model README
<!-- D0XIN_RELEASE_NAV_END -->Related models: all models
Swift-1.5-Qwen3.8-27B-Uncensored-BF16 · Swift-1.5-Qwen3.8-27B-Uncensored-FP8 · Swift-1.5-Qwen3.8-27B-Uncensored-FP8-NInfer · Swift-1.5-Qwen3.8-Flash-Next-NVFP4-FP8PLE · Swift-1.5-Qwen3.8-Flash-Next-Rank2-Abliteration-Patch · Swift-Qwen3.8-27B-FP8 · Swift-Qwen3.8-27B-Uncensored-BF16 · Swift-Qwen3.8-27B-Uncensored-FP8 · Swift-Qwen3.8-27B-Uncensored-NVFP4-LocalHessian-ActivationHeadroom-NInfer
Native NInfer build of d0xin/Swift-Qwen3.8-27B-Uncensored-BF16, re-quantized from BF16 using NVIDIA ModelOpt Local-Hessian calibration + activation headroom and a conservative mixed NVFP4/FP8 topology.
The goal of this release is straightforward: retain Swift-Qwen3.8's capabilities while removing refusal behavior.
It is designed to preserve Swift-Qwen3.8's reasoning, agentic, tool-calling, multimodal and long-context capabilities while removing refusal behavior.
For this quantized build specifically, the goal is also to increase inference throughput and reduce VRAM usage while preserving FP8-level measured quality and avoiding longer reasoning traces under the recommended B2048 serving profile.
In our fixed validation, this build delivered +20–27% decode throughput, +34–38% prefill throughput, and 15.1% lower VRAM usage versus FP8, while scoring 225/280 vs 224/280 on the B2048 MMLU-Pro subset. Mean reasoning usage was also lower: 741.9 vs 793.0 tokens.
This is not a repack of an existing NVFP4 checkpoint.
| Component | Precision |
|---|---|
| MLP layers 0-55 | NVFP4 W4A4 |
| MLP layers 56-63 | row-wise FP8 |
| Self-attention projections | row-wise FP8 |
| Linear-attention / GDN projections | row-wise FP8 |
| GDN a/b | BF16 |
| LM head | row-wise FP8 |
| Embedding | row-wise FP8 |
Calibration used NVIDIA ModelOpt Local-Hessian, activation headroom, FP8 scale sweep, layer-wise checkpointing, upper_percentile=99.99, and rho=16384.
Fixed MMLU-Pro subset: 280 questions, 14 categories, 20 questions per category, seed 20260918, temperature=0, reasoning_effort=xhigh, max_tokens=4096.
| Build | Correct | Accuracy |
|---|---|---|
| FP8 | 211/280 | 75.36% |
| NVFP4 Headroom | 207/280 | 73.93% |
| FP8 + B2048 | 224/280 | 80.00% |
| NVFP4 Headroom + B2048 | 225/280 | 80.36% |
For FP8+B2048 vs this build+B2048: both correct 216, both wrong 47, FP8-only correct 8, NVFP4-only correct 9, exact McNemar p=1.0.
No statistically significant accuracy difference was observed on this subset.
Recommended NInfer setting: --default-thinking-budget 2048.
Without a thinking budget, 36/280 NVFP4 evaluation generations reached the 4096-token output ceiling. With B2048, reasoning median was 379.5 tokens, p90 was 2072, mean was 741.87, 0/280 reached the 4096-token limit, all 280 requests finished normally, and accuracy was 225/280.
The thinking budget is a serving recommendation and is not baked into the artifact.
Test hardware: NVIDIA RTX PRO 6000 Blackwell Workstation Edition.
| Metric | FP8 | This build | Change |
|---|---|---|---|
| Decode C1 | 134.87 tok/s | 162.32 tok/s | +20.4% |
| Decode C8 | 87.33 tok/s | 110.58 tok/s | +26.6% |
| Prefill C1 | 6633.88 tok/s | 8876.31 tok/s | +33.8% |
| Prefill C8 | 6960.00 tok/s | 9615.82 tok/s | +38.2% |
| NInfer VRAM | 41358 MiB | 35106 MiB | -15.1% |
Performance is hardware- and workload-dependent.
Fixed Swift ORCA-K1 100-prompt evaluation:
| Classification | Count |
|---|---|
| DIRECT | 90 |
| SAFETY_DEFLECT | 10 |
| REFUSE | 0 |
| OTHER_FAILURE | 0 |
Configuration: temperature=0, seed=42, max_tokens=1536, thinking disabled.
Historical FP8 responses re-judged with the same current judge produced 87 DIRECT, 12 SAFETY_DEFLECT, 1 REFUSE, and 0 OTHER_FAILURE.
The 0/100 refusals result applies only to this fixed automated evaluation. It is not a universal non-refusal guarantee.
This model has deliberately reduced refusal behavior.
Reduced refusal behavior also reduces model-level safety filtering. Users are responsible for evaluating the model for their own deployment, safety, legal, and compliance requirements.
The 0/100 figure describes the documented fixed evaluation set and
inference configuration. It should not be interpreted as a guarantee that
no conceivable prompt, sampling configuration, system prompt, or inference
engine can ever produce a refusal.
Swift-Qwen3.8-27B-Uncensored-NVFP4-LH-ActHeadroom.ninfer23,719,715,844 bytesa298e28529a31f5447574ba2ad5ba5812d952053292ca017115f9d735dfdadcd1,246The artifact contains Text, Vision, MTP, DFlash2, the optimized proposal head, and tokenizer/frontend resources.
Validated NInfer revision: f76e19c0fbd026c86f46005acf2c80c54084bade.
Current NInfer targets NVIDIA Blackwell sm_120a.
This build was directly validated on an NVIDIA RTX PRO 6000 Blackwell Workstation Edition.
The artifact is expected to fit on a 32 GB RTX 5090, but the exact RTX PRO 6000 serving profile should not be copied unchanged.
Our RTX PRO 6000 production profile uses about 35.1 GiB VRAM, so the full --kv-capacity 262144 profile with Vision enabled does not fit within 32 GB.
Upstream NInfer runs Qwen3.8-27B NVFP4 on RTX 5090 with approximately:
This exact quantized build has not yet been directly benchmarked by the author on RTX 5090, so RTX 5090 support should be considered expected rather than independently validated.
The current .ninfer runtime should not be considered portable to Ampere, Ada, or Hopper.
For a 32 GB RTX 5090, start with a text-only profile like this:
ninfer-serve \
./Swift-Qwen3.8-27B-Uncensored-NVFP4-LH-ActHeadroom.ninfer \
--host 0.0.0.0 \
--port 8080 \
--model-id Swift-Qwen3.8-27B-Uncensored-NVFP4 \
--max-context 250000 \
--kv-capacity auto \
--max-concurrency 2 \
--kv-dtype fp8 \
--prefill-chunk 1024 \
--spec dflash2 \
--draft-tokens 5 \
--lm-head-draft \
--preserve-thinking \
--default-thinking-budget 2048
For Vision workloads, use a substantially smaller context budget; 81920 is a reasonable starting point based on upstream NInfer's RTX 5090 Qwen3.8 NVFP4 profile.
--kv-capacity auto is recommended on 32 GB cards so NInfer can size the shared Main Text KV pool from the VRAM remaining after model and runtime allocations.
ninfer-serve \
./Swift-Qwen3.8-27B-Uncensored-NVFP4-LH-ActHeadroom.ninfer \
--host 0.0.0.0 \
--port 8080 \
--model-id Swift-Qwen3.8-27B-Uncensored-NVFP4 \
--max-context 262144 \
--kv-capacity 262144 \
--max-concurrency 8 \
--kv-dtype fp8 \
--prefill-chunk 1024 \
--spec dflash2 \
--draft-tokens 5 \
--lm-head-draft \
--preserve-thinking \
--default-thinking-budget 2048 \
--vision
--max-context 262144 is the logical ceiling for one request.
--kv-capacity 262144 is a shared Main Text KV pool across concurrently admitted requests. It does not allocate 262144 KV tokens independently to each concurrent request.
f76e19c0fbd026c86f46005acf2c80c54084bade9e3d555aa16038f7b46a631ca603351146c091236693441a7db6984415d524f51ed16a1605bf2fec0.18.1.a20260914Qwen/Qwen3.8-27B
↓
ukisai/Swift-Qwen3.8-27b
↓
d0xin/Swift-Qwen3.8-27B-Uncensored-BF16
↓
Local-Hessian + Activation Headroom
mixed NVFP4/FP8
↓
this NInfer artifact
Quantization can change individual generation trajectories even when aggregate benchmark accuracy is similar.
The MMLU-Pro result reported here uses a fixed 280-question subset, not the full benchmark. The refusal evaluation uses a fixed automated 100-prompt set. Performance results were measured on one RTX PRO 6000 Blackwell system and should not be interpreted as universal performance values.
This model is a derivative of ukisai/Swift-Qwen3.8-27b.
Swift-specific contributions are distributed under the Swift Open License v1.0. The underlying Qwen base model is distributed under Apache License 2.0.
See the included LICENSE, LICENSE-APACHE-2.0, and NOTICE files for applicable terms and attribution requirements.
Qwen/Qwen3.8-27Bukisai/Swift-Qwen3.8-27bd0xin