LiveNew tools this week

Nemotron-Cascade-2-30B-A3B-NVFP4

wyattearp/Nemotron-Cascade-2-30B-A3B-NVFP4

Nemotron-Cascade-2-30B-A3B-NVFP4 is a machine learning model from wyattearp. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.

Self-quantized on an NVIDIA DGX Spark (GB10 Blackwell) using NVIDIA TensorRT Model Optimizer (main branch, 0.47.0.dev41+g43e9d15e0) PTQ, from the original BF16 checkpoint nvidia/Nemotron-Cascade-2-30B-A3B.

Downloads · 30 days

15

4% of all-time downloads

All-time downloads

383

Public

Parameters

16.9B

19.6 GB on disk

Likes

0

Public

Repo makeup

Click a slice to open those files.

.safetensors19.6 GB · 100%

Parameter types

How the weights are stored.

U814.7B · 87%

Base models

nvidia/Nemotron-Cascade-2-30B-A3B

Model card

From the Hugging Face model README

Nemotron-Cascade-2-30B-A3B — NVFP4 (Mixed Precision)

Self-quantized on an NVIDIA DGX Spark (GB10 Blackwell) using NVIDIA TensorRT Model Optimizer (main branch, 0.47.0.dev41+g43e9d15e0) PTQ, from the original BF16 checkpoint nvidia/Nemotron-Cascade-2-30B-A3B.

Quantization Recipe

Adapted from ModelOpt's huggingface/models/nvidia/Nemotron-3-Super-120B-A12B/ptq/nvfp4-mse.yaml (a sibling NemotronH-family model's recipe):

Component	Precision	Rationale
MoE routed experts (`mixer.experts.*`)	NVFP4 (W4A4, group_size 16)	MSE-calibrated static block scales.
MoE shared experts (`mixer.shared_experts.*`)	FP8 per-tensor	Shared MLP throughput path.
Mamba mixer linears (`mixer.{in,out}_proj`)	FP8 per-tensor	Preserves linear projections.
Mamba SSM Kernels & RMSNorms	BF16 / FP32	Preserves continuous recurrence scan state (`A_log`, `D`, `conv1d`, `dt_bias`).
Attention linears & KV Cache	Native BF16	Preserves RoPE geometry and eliminates uncalibrated FP8 attention q_scale degradation.

Calibration: 4,000 samples (1,000 each from nemotron-sft-agentic-v2, nemotron-sft-swe-v2, nemotron-math-v2, nemotron-science-v1), weight-MSE calibration, batch size 8, 500 steps.

🛠️ Serving Instructions with vLLM (GB10 / GB200 Blackwell)

[!IMPORTANT] Use qwen3_xml, NOT qwen3_coder: Empirical evaluation on multi-turn benchmarks demonstrated that qwen3_coder uses a fragile Python regex parser that fragments streaming buffers and drops reasoning tokens during multi-line shell execution (<, >, |). Switching to qwen3_xml (Python's C-based streaming Expat XML state machine) increases agentic exploit derivation hit rates from 20% to 50%.

docker run -d --gpus all --network inference-net --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 \
  -e VLLM_USE_FLASHINFER_MOE_FP4=1 \
  -e VLLM_FLASHINFER_MOE_BACKEND=throughput \
  -v /path/to/this/checkpoint:/model \
  vllm/vllm-openai:latest \
  /model \
  --served-model-name nvidia/nemotron-cascade-2-30b-nvfp4 \
  --trust-remote-code \
  --mamba-ssm-cache-dtype float32 \
  --reasoning-parser nemotron_v3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --gpu-memory-utilization 0.75 \
  --max-model-len 262144

🔬 Empirical Findings & Benchmarks

1. Generation Throughput (DGX Spark GB10, vLLM 0.20.0)

Prompt	BF16	NVFP4 (This Checkpoint)	Speedup
General Knowledge	28.66 tok/s	35.36 tok/s	1.23×
Math Reasoning	27.77 tok/s	45.26 tok/s	1.63×
Coding Task	28.52 tok/s	53.47 tok/s	1.87×
Agentic Tool-Call	27.87 tok/s	51.13 tok/s	1.83×
Average	~28.2 tok/s	~46.3 tok/s	~1.64×

2. Multi-Turn Tool Parser Impact (`qwen3_coder` vs. `qwen3_xml`)

Configuration	Hit Rate (n=10)	Behavior
Cascade-2 NVFP4 + `qwen3_coder`	20%	Regex slices streaming chunks; drops `<think>` reasoning and emits orphaned character debris (`e`, `s`, `r`).
Cascade-2 NVFP4 + `qwen3_xml`	40% – 50%	Expat XML streaming parser maintains full state; parses heredocs, bash redirects, and reasoning streams cleanly.

🔧 Upstream Configuration Fixes Applied to this Checkpoint

Native KV Cache Stability: Stripped uncalibrated kv_cache_scheme: fp8 declaration so KV cache defaults to native BF16, preventing the Using uncalibrated q_scale 1.0 with fp8 attention accuracy degradation.
Topology Metadata: Corrected layers_block_type, removed orphaned mtp_layers_block_type, and explicitly declared num_hidden_layers: 52.

Type: nemotron_h
License: other
Created: Aug 7, 2026
Updated: Aug 20, 2026