NVIDIA-Nemotron-3.5-Lightning-30B-A3B-DSpark-bf16

NVIDIA-Nemotron-3.5-Lightning-30B-A3B-DSpark (bf16, MLX)

The DSpark speculative-decoding drafter for nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B, dequantized to bf16 for use on Apple Silicon with mlx-dspark.

This is a format conversion, not a retrain. NVIDIA ships the drafter as nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark in NVIDIA Model Optimizer NVFP4 (W4A16_NVFP4), which stock MLX / mlx-lm cannot read. The FP4 (E2M1) weights + FP8 (E4M3) block scales were hand-decoded back to bf16 (attention / embed_tokens / markov_w1 were already bf16 in the source). Weights are otherwise unchanged; mlx-dspark re-quantizes the drafter to 4-bit at load, which does not affect acceptance.

Use with mlx-dspark

Pair it with the 4-bit MLX target mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-4bit:

pip install mlx-dspark
mlx-dspark generate \
  --model mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-4bit \
  --drafter mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-DSpark-bf16 \
  --max-draft 4 --prompt "Write a Python function for the n-th Fibonacci number."

The output is lossless — the target verifies every token, so speculation only changes speed, never the text (it matches greedy target decoding up to floating-point ties).

Architecture

A plain Qwen3 GQA drafter backbone (6 layers, hidden 2688, 32 heads / 2 KV, head_dim 128, vocab 131072) with the DSpark rank-512 Markov fixup head and DFlash-lineage traits: causal sliding-window (1024) block attention with a per-head attention-sink bias, block_size 8, sample_from_anchor=false (predictions start at block slot 1). It ships its own embed_tokens and reuses the target's lm_head (has_lm_head=false). 967M params total (615M non-embedding). Fuses 6 target hidden-state taps (layers 1, 5, 19, 29, 41, 51).

The target is a hybrid Mamba-2 + latent-MoE + attention model (model_type: nemotron_h). mlx-dspark adds the nemotron_h hidden-state tap and an exact Mamba-2 recurrent-state rollback so the lossless verify loop works on it.

Measured (M4 Pro, mlx 0.32, greedy, warm)

Baseline (4-bit target) ≈ 91 tok/s. DSpark at --max-draft 4 (lookup off):

Throughput peaks at cap 4 and regresses past it (the MoE verify-width cost climbs faster than acceptance) — the speedup here is bounded by the MoE, not the drafter, whose acceptance is strong.

License & attribution

Original drafter © NVIDIA, released under the OpenMDW-1.1 model license (see LICENSE). This repository redistributes a dequantized (NVFP4 → bf16) copy under the same terms. Produced with NVIDIA Model Optimizer (source) and mlx-dspark (conversion).