Downloads · 30 days
26
5% of all-time downloads
soyrsoyr/Nemotron-3.5-Lightning-1.4B-A0.1B-MTP-NVFP4
Nemotron-3.5-Lightning-1.4B-A0.1B-MTP-NVFP4 is a text generation model from soyrsoyr. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
This is an architecture test fixture derived from inference-optimization/Nemotron-3.5-Lightning-1.4B-A0.1B-MTP. Its MTP tensors use the current NemotronHForCausalLM layout consumed by vLLM: one full-attention block fo…
Downloads · 30 days
26
5% of all-time downloads
All-time downloads
492
Public
Parameters
1B
5.7 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors2.1 GB · 99%
How the weights are stored.
BF161B · 100%
From the Hugging Face model README
This is an architecture test fixture derived from
inference-optimization/Nemotron-3.5-Lightning-1.4B-A0.1B-MTP. Its MTP tensors
use the current NemotronHForCausalLM layout consumed by vLLM: one
full-attention block followed by one MoE block. The backbone remains BF16 so the
fixture isolates MTP compression behavior.
The source mini checkpoint used a legacy flattened MTP layout. For this fixture, the MTP blocks were rebuilt from shape-compatible full-attention and MoE backbone blocks before running LLM Compressor. This model is for structural and serving tests, not quality evaluation.
from llmcompressor import oneshot
oneshot(
model="/path/to/Nemotron-3.5-Lightning-1.4B-current-MTP-layout",
recipe=[],
output_dir="/workspace/models/Nemotron-3.5-Lightning-1.4B-A0.1B-MTP-NVFP4",
mtp_scheme="NVFP4",
precision="bfloat16",
)
mtp_scheme is handled during oneshot post-processing. Because Transformers
does not construct the MTP modules for calibration, the MTP group uses
weight-only NVFP4; no MTP activation scales are stored.
model_mtp.safetensors.vllm serve soyrsoyr/Nemotron-3.5-Lightning-1.4B-A0.1B-MTP-NVFP4 \
--speculative-config '{"method":"mtp","num_speculative_tokens":1}'