Downloads · 30 days
63
12% of all-time downloads
Seidkona-Dreams/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-oQ4-mtp
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-oQ4-mtp is a machine learning model from Seidkona-Dreams. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as other.
This is nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, quantized to MLX format with oQ (oMLX) mixed-precision quantization, with the model's native Multi-Token-Prediction (MTP) head preserved (preservemtp=True) ra…
Downloads · 30 days
63
12% of all-time downloads
All-time downloads
521
Public
Parameters
32.9B
19.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors19.4 GB · 100%
How the weights are stored.
U3232.9B · 100%
From the Hugging Face model README
This is nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16,
quantized to MLX format with oQ (oMLX) mixed-precision quantization, with the model's
native Multi-Token-Prediction (MTP) head preserved (preserve_mtp=True) rather than stripped, so it can drive
self-speculative decoding ("Lightning MTP" in oMLX) instead of plain autoregressive decoding.
model_type=nemotron_h)num_nextn_predict_layers=1, native mtp.* tensors kept, not stripped)The MTP head is loaded and driven by oMLX's mlx_lm_mtp runtime patches (depth-k drafting for the nemotron_h
hybrid Mamba/attention trunk). This is not vanilla mlx-lm behavior — loading this checkpoint with plain
mlx-lm/mlx-vlm will work for standard generation, but the MTP head will only be exercised for speculative
decoding under oMLX (or another runtime that implements an equivalent nemotron_h MTP chain). Greedy speculative
decoding with the head is lossless: output tokens are identical to the depth-1/no-MTP greedy stream, only
throughput changes.
Informal spot-check under oMLX with Lightning MTP enabled (single warm request, not a full benchmark suite): ~87% MTP draft-token acceptance rate, ~62 tok/s output on Apple Silicon (M-series).
This model is a derivative of NVIDIA's Nemotron-3.5-Lightning-30B-A3B, distributed under the
OpenMDW-1.1 license. Per the license terms, this distribution retains a
reference to the license and to NVIDIA's original copyright/origin notices. See the base model's
model card and datasets
(nvidia/nemotron-post-training-v3, nvidia/nemotron-pre-training-datasets) for full provenance.
THE MODEL MATERIALS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND. See the license link above for the full text and terms.