ThinkingCap-Qwen3.6-27B-Abliterix-FP8

ThinkingCap Qwen3.6 27B Abliterix — FP8

NVIDIA ModelOpt FP8 unified Hugging Face checkpoint produced from the ThinkingCap Qwen3.6-derived Abliterix checkpoint. The checkpoint declares the Qwen3_5ForConditionalGeneration architecture and is intended for vLLM on Blackwell-class NVIDIA hardware.

This is an abliterated model. Treat its safety behavior and outputs as altered; evaluate it for your own deployment.

Quantization

• ModelOpt: 0.45.0
• PyTorch / CUDA: 2.11.0 / CUDA 13.0
• Transformers: 5.9.0
• Format: ModelOpt FP8
• Calibration: 256 instruction conversations from HuggingFaceH4/ultrachat_200k and 128 image-question examples from the AI2D subset of HuggingFaceM4/the_cauldron
• Text calibration length: 2,048 tokens; calibration images are resized to at most 1,500,000 pixels without truncating multimodal placeholder tokens
• KV cache: BF16/default (kv_cache_quant_algo: null)
• Execution hardware: two NVIDIA B200 GPUs; intended runtime: vLLM on Blackwell hardware, including DGX Spark / GB10-class deployments

Features

• Text, image and video processor assets are included.
• The vision/video tower and merger remain BF16, as do embeddings, normalizations, LM head, hybrid linear-attention modules, and MTP-sensitive paths.
• The source configuration does not contain an audio tower or audio weights; audio capability is therefore not claimed.
• The underlying Qwen MTP configuration is preserved. Enable self-speculation only when supported by your installed vLLM build.

No benchmark or quality claims are made here; evaluate the quantized checkpoint on your own representative text and multimodal workloads.

vLLM

vllm serve edougawa/ThinkingCap-Qwen3.6-27B-Abliterix-FP8 \
  --quantization modelopt \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

For a non-speculative launch, omit --speculative-config.