Downloads · 30 days
309
44% of all-time downloads
HangGlidersRule/Darkstar-Nemotron-3.5-Lightning-30B-A3B-Base-ModelOpt-W4A16-NVFP4
Darkstar-Nemotron-3.5-Lightning-30B-A3B-Base-ModelOpt-W4A16-NVFP4 is a text generation model from HangGlidersRule. Use it when you need the model to write or continue text. The card lists the license as other.
Clean, unedited NVIDIA ModelOpt quantization of nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 at revision d468880b6ad3c6e0d21377ce7242adaea4cc884d — the control artifact for the Darkstar Nemotron-3.5-Lightning abl…
Downloads · 30 days
309
44% of all-time downloads
All-time downloads
709
Public
Parameters
16.7B
23 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors22.9 GB · 100%
How the weights are stored.
U814.9B · 83%
From the Hugging Face model README
Clean, unedited NVIDIA ModelOpt quantization of
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
at revision d468880b6ad3c6e0d21377ce7242adaea4cc884d — the control artifact for the Darkstar
Nemotron-3.5-Lightning abliterated derivatives: same source revision, same quantization contract, no
refusal-direction projection. The mixed layout quantizes routed + shared expert up/down projections
(5,934 modules) to W4A16-NVFP4 group 16 and keeps lm_head, Mamba/SSM (conv1d, in_proj, out_proj,
A_log, D, dt_bias), attention (q/k/v/o + BMM), norms, embeddings, and the MTP head protected in BF16.
Artifact is 3 shards, ≈22 GB; quantization_config maps to vLLM modelopt_mixed.
ModelOpt is pinned to 0.46.0rc2 at
43fd41a58d52c4e6e5dec1d1ff5989ecc737ae1a. Calibration used cnn_dailymail plus
nvidia/Nemotron-Post-Training-Dataset-v2, 512+512 samples, sequence length 2048, seed 1234, batch 1,
KV-cache quantization disabled (BF16).
Full provenance, protocol, and caveats:
| Metric | Value | Basis |
|---|---|---|
| Behavior: harmful-prompt compliance | not measured (unedited control) | refusal behavior unchanged by design; no projection applied |
| Safe over-refusals | not measured (unedited control) | same basis |
| Single-stream throughput (MTP7) | 541.7 tok/s weighted | 4K/16K/48K = 562.2/548.0/399.7; sweep winner; DFlash 523.9 |
Missing cells are marked not measured and are never backfilled from a different checkpoint or
protocol.
This is the unedited control quantization. The refusal measurements are behavior observations, not a safety endorsement. Quantization can shift behavior relative to upstream BF16. Only single-stream throughput is characterized here; aggregate concurrent throughput is separate.
Engineering release: darkstar-nemotron-3.5-lightning-v1.0.0. This immutable tag exists and the release contract is published.
Validated with vLLM (CUDA 13 Blackwell nightly build family), Flash Attention, BF16 KV cache,
context 131,072, MTP depth 7, and max_num_seqs=16:
vllm serve HangGlidersRule/Darkstar-Nemotron-3.5-Lightning-30B-A3B-Base-ModelOpt-W4A16-NVFP4 \
--served-model-name darkstar-nemotron35lightning-base-nvfp4 \
--kv-cache-dtype bfloat16 \
--max-model-len 131072 \
--max-num-seqs 16 \
--reasoning-parser nemotron_v3 \
--speculative-config '{"method": "mtp", "num_speculative_tokens": 7}'