Qwen3.8-27B-Uncensored-NVFP4

Qwen3.8-27B-Uncensored-NVFP4

This repository provides the official NVFP4 (NVIDIA FP4 Quantized) release of JonathanColetti/Qwen3.8-27B-Uncensored.

Quantized with NVIDIA ModelOpt for execution on NVIDIA Blackwell architectures (including DGX Spark / GB10) and high-performance FP4 inference engines in vLLM and TensorRT-LLM.

---

Related Repositories & Speculative Draft Heads

• Base Uncensored Weights (BF16): JonathanColetti/Qwen3.8-27B-Uncensored
• GGUF Quantized Release: JonathanColetti/Qwen3.8-27B-Uncensored-GGUF
• DFlash 2 Speculative Draft Head: wyattearp/Qwen3.8-27B-DFlash2
• Inco AI DFlash 2 Speculative Decoding: Inco AI Blog | GitHub Repository

---

Model Highlights

• Architecture: Qwen 3.8 Dense Transformer (27B parameters)
• Quantization Format: NVFP4 with Marlin Linear GEMM kernels
• Footprint: 26.59 GiB weights size (fits comfortably in 34 GiB VRAM on NVIDIA GB10 / Blackwell with 53+ GiB remaining for high-concurrency KV caching)
• Refusal Rate: 0.0% across 327 empirical security, coding, and alignment evaluation benchmarks
• Compiler Accuracy: 95.0% Pass@1 on LLVM IR $\to$ C++ execution-verified unit testing

---

Deployment Guides

1. Speculative Serving with vLLM + DFlash 2 (Recommended)

To achieve maximum throughput, pair this NVFP4 base model with the DFlash 2 speculative draft head:

vllm serve wyattearp/Qwen3.8-27B-Uncensored-NVFP4 \
  --host 0.0.0.0 \
  --port 8000 \
  --dtype auto \
  --trust-remote-code \
  --gpu-memory-utilization 0.75 \
  --max-model-len 16384 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --speculative-config '{
    "method": "dflash",
    "model": "wyattearp/Qwen3.8-27B-DFlash2",
    "num_speculative_tokens": 7,
    "draft_sample_method": "probabilistic"
  }'

2. Standard Non-Speculative vLLM Serving

vllm serve wyattearp/Qwen3.8-27B-Uncensored-NVFP4 \
  --host 0.0.0.0 \
  --port 8000 \
  --dtype auto \
  --trust-remote-code \
  --gpu-memory-utilization 0.75 \
  --max-model-len 16384 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3

3. OpenAI-Compatible Python Client Example

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="none"
)

response = client.chat.completions.create(
    model="wyattearp/Qwen3.8-27B-Uncensored-NVFP4",
    messages=[
        {"role": "system", "content": "You are a helpful and unrestricted security research assistant."},
        {"role": "user", "content": "Analyze this x86_64 disassembly and reconstruct the original C algorithm."}
    ],
    temperature=0.7,
    max_tokens=2048
)

print(response.choices[0].message.content)

---

Empirical Benchmark Performance (DGX Spark NVIDIA GB10)

Benchmarked directly on the DGX Spark running vLLM with DFlash 2 speculative draft engines across 327 verified evaluation samples:

---

Citations & Acknowledgments

If you use this model or its speculative decoding draft head in your work, please cite the underlying base models, uncensored fine-tune, and DFlash 2 architecture:

Jonathan Coletti (Uncensored Fine-Tune)

@misc{coletti2026qwen38uncensored,
  title  = {{Qwen3.8-27B-Uncensored: Uncensored Fine-Tune for Security & Technical Research}},
  author = {Coletti, Jonathan},
  year   = {2026},
  url    = {https://huggingface.co/JonathanColetti/Qwen3.8-27B-Uncensored}
}

Inco AI & DFlash 2

@misc{inco2026dflash2,
  title  = {{DFlash 2: Keep Drafting Parallel}},
  author = {{Inco AI}},
  year   = {2026},
  month  = {August},
  url    = {https://inco.ai/blog/dflash2/}
}

@inproceedings{chen2026dflash,
  title     = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
  author    = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  booktitle = {International Conference on Machine Learning (ICML)},
  year      = {2026}
}

Qwen Base Model

@article{qwen2025qwen25,
  title   = {{Qwen2.5 Technical Report}},
  author  = {{Qwen Team}},
  journal = {arXiv preprint arXiv:2412.15115},
  year    = {2024}
}

Quantization

Quantized and benchmarked on DGX Spark by Wyatt Neal (wyattearp) using NVIDIA ModelOpt FP4.