Downloads · 30 days
117
56% of all-time downloads
HangGlidersRule/Darkstar-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-Abliterated-ModelOpt-W4A16-NVFP4
Darkstar-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-Abliterated-ModelOpt-W4A16-NVFP4 is a image-text-to-text model from HangGlidersRule. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as other.
Reduced-refusal model: a refusal-behavior edit was deliberately applied in BF16 before quantization. Read the safety warning before use.
Downloads · 30 days
117
56% of all-time downloads
All-time downloads
208
Public
Parameters
33B
42.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors22.6 GB · 100%
How the weights are stored.
U830.2B · 91%
From the Hugging Face model README
Reduced-refusal model: a refusal-behavior edit was deliberately applied in BF16 before quantization. Read the safety warning before use.
model-forge#35. The weights published as v1.0.0 were a bare NemotronHForCausalLM language trunk: no vision or audio towers, so the model could not take image or audio input despite the image-text-to-text tag. Its 23 MoE routers' e_score_correction_bias tensors had also been rounded through BF16. v1.1.0 is the full NemotronH_Nano_Omni_Reasoning_V3 wrapper:
nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16;NVIDIA ModelOpt quantization of the Darkstar Abliterated BF16 derivative of
nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16,
pinned upstream at 24e67ea000b7c2837fc8f9488aa2008524fac8ba. The edit is the r1 LoRA
unlearning SFT (rank-32 adapter on the language trunk's q/k/v/o_proj, α=64, teacher = the
model's own compliant rows under the campaign teacher-selection boundary; see the
Abliterated-BF16 card
for the full method contract and the closed-form projection rejection evidence). The edited BF16
artifact was then quantized: MoE expert projections (routed + shared, 5,894 modules) and lm_head
to W4A16-NVFP4 group 16; Mamba2/SSM path, attention, norms, embeddings, and the vision/audio
towers protected in BF16.
ModelOpt is pinned to 0.46.0rc2 at 43fd41a58d52c4e6e5dec1d1ff5989ecc737ae1a with an MSE
calibration algorithm (fp8_scale_sweep). Calibration used cnn_dailymail plus
nemotron-post-training-dataset-v2, 512+512 samples, sequence length 2048, seed 1234, batch 1,
KV-cache quantization disabled (BF16). Quantization ran on the language trunk extracted from the
VLM wrapper as a standalone plain nemotron_h checkpoint; the wrapper was then reassembled byte for
byte around the quantized trunk (see v1.1.0 repair), with the towers identical to the edited BF16 parent.
Full provenance, protocol, and caveats:
| Metric | Value | Basis |
|---|---|---|
| GPQA Diamond (thinking on) | 67.5/198 = 34.1% (two runs: 64 and 71) | llm-inference-bench gpqa-diamond, chat template + thinking ON, temp 0, full 198 denominator, 8,192-token answer cap, concurrency 30 (the v1.0.0 protocol); vLLM v0.32.0 (pinned digest befccf0e, plus the vllm[audio] extras), one RTX PRO 6000 Blackwell; 0 errors (run 1, run 2) |
| Abliterated-BF16 parent, same session (one run) | 70/198 = 35.4% | same protocol and runtime (result); quantization cost within noise |
| Published v1.0.0 (historical) | 58/198 = 29.3% | vLLM 0.27.1 (aeon build), BF16 KV cache, trunk-only weights |
| Behavior: harmful-prompt compliance | 200/200 (0/200 refusals) | measured on the repaired weights, served; 0 errors (result) |
| Safe over-refusals | 0/83 (0.00%) | same run; 0 errors |
| Image and audio input | 14/14 images and 1/1 speech clip answered | serving window on the repaired weights; a clean download of the staged v1.1.0 files also answered 8 text, 2 image and 1 speech request (result) |
| Single-stream throughput (no spec decode) | 249.468 tok/s weighted (v1.0.0 weights, vLLM 0.27.1) | 4K 248.64 / 16K 250.80 / 48K 250.43; not re-measured on the repaired weights |
Missing cells are marked not measured and are never backfilled from a different checkpoint or
protocol.
Intelligence cost: on the same runtime, this checkpoint scores 64 and 71 in two runs and its BF16 parent 70, so the quantization costs nothing measurable at this run-to-run spread. NVIDIA's own NVFP4 of the unedited model scored 115/198 (result) on the same GPU and harness at a 65,536-token cap, so the edit and this quantization recipe carry a real reasoning cost; do not use this artifact for production reasoning work.
This model has had its refusal behavior deliberately reduced. It complied with 200/200 harmful prompts in the measured suite and has no added safety mitigations. It will comply with many requests the upstream model would refuse. Deploy only behind appropriate policy, filtering, access controls, and legal review. Refusal-rate numbers are behavior measurements, not safety endorsements.
Family release: darkstar-nemotron-3-nano-omni-v1.1.0. This immutable tag exists and the release contract is published.
Validated with vLLM 0.32.0 plus the audio extras (pip install "vllm[audio]"), MARLIN NvFp4 MoE backend (vLLM's default for this layout), KV cache in the checkpoint dtype, context 131,072. --mamba-ssm-cache-dtype float32 is required: vLLM does not set it for the Omni wrapper, and the default lowers quality.
vllm serve HangGlidersRule/Darkstar-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-Abliterated-ModelOpt-W4A16-NVFP4 \
--served-model-name darkstar-nemotron3nanoomni-abliterated-nvfp4 \
--trust-remote-code \
--max-model-len 131072 \
--max-num-seqs 16 \
--mamba-ssm-cache-dtype float32 \
--reasoning-parser nemotron_v3 \
--gpu-memory-utilization 0.85