Signal-3.8-27b-Heretic-ara-NVFP4

Signal-3.8-27b-Heretic-ara-NVFP4

NVFP4 (W4A16) quantization of Shockem/Signal-3.8-27b-Heretic-ara — an abliterated ("heretic") variant of agentionai/Signal-3.8-27B — for NVIDIA Blackwell GPUs (RTX 50-series, DGX Spark, B100/B200). Produced with NVIDIA TensorRT Model Optimizer 0.45.0. Intended for research purposes.

The BF16 source was abliterated with p-e-w/heretic (100-trial Optuna study; winning recipe merged onto pristine BF16 in fp32). See the BF16 repo for the full study notes. This repo is the same weights, quantized to NVFP4.

Format

• Same export format as other modelopt NVFP4 checkpoints of this arch: weight (U8-packed FP4), weight_scale (F8_E4M3, block 16), weight_scale_2 (F32) per quantized tensor; 400 quantized tensors, everything else BF16 passthrough.
• Vision tower and in-checkpoint MTP head preserved, plus a vocab-truncated 40960-id BF16 draft head (mtp.draft_lm_head.weight, mtp_draft_vocab_ids.pt) for faster speculative decoding with a patched qwen3_5_mtp.py (see syv-ai's draft-vocab work; stock vLLM ignores it harmlessly).

Sampling

Follow the base model's updated generation_config.json (already included here): temperature 0.6, top_p 0.95, top_k 20, min_p 0.05. Prefer sampling over greedy.

Serving

Tested with vllm/vllm-openai:v0.28.0:

vllm serve Shockem/Signal-3.8-27b-Heretic-ara-NVFP4 \
  --quantization modelopt --dtype bfloat16 --kv-cache-dtype fp8_e4m3

House test results (producer's rig)

Measured on the producer's stack (vllm/vllm-openai:v0.28.0, TP=2 on 2× RTX 5060 Ti 16 GB, 200k context, FP8 KV cache, MTP 3 + the 40960-id draft head, CUDA graphs):

• Coding harness (14-test agentic suite, 10 repeats): easy 100% · medium 100% · hard 98.7%. The single miss is a known LRU-cache codegen flake (8/10) seen on other checkpoints of this architecture too.
• Long-prose generation: ~56 tok/s wall with ~41% speculative acceptance.

Serving note: the producer runs repetition_penalty: 1.05 as the served default instead of min_p, because vLLM 0.28 rejects min_p when speculative decoding is enabled. Both guard the same runaway-repetition failure mode the base model's card warns about; if your server accepts min_p with spec decode, use the shipped generation_config.json as-is.

Optional: FP8 draft stack (+6.6% decode)

The MTP draft stack (mtp.* is excluded from the NVFP4 quantization, so it normally runs as BF16) can be converted to FP8 e4m3 for a measurable decode speedup with no output-quality change — the drafter only proposes tokens; the target model verifies every one. Measured on the producer's rig (2× RTX 5060 Ti, vLLM 0.28.0, TP=2, 200k ctx, FP8 KV, MTP 3, batch 1):

Converter script, patched qwen3_5_mtp.py (vLLM 0.28.0), and instructions: FP8 MTP draft stack gist. Note: the FP8 variant checkpoint requires the patched model file — stock vLLM will silently misload it, so keep the stock checkpoint as fallback.

Caveats

• Abliteration suppresses refusal behavior with minimal — but not zero — capability impact. For the original behavior, use agentionai/Signal-3.8-27B.
• This is a research artifact. You are responsible for how you use it.

Credits

• Base model: agentionai/Signal-3.8-27B
• Abliteration tool: p-e-w/heretic
• Quantization: NVIDIA TensorRT Model Optimizer 0.45.0