Downloads · 30 days
55
19% of all-time downloads
meshllm/NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL-MTPv2-GGUF
NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL-MTPv2-GGUF is a text generation model from meshllm. Use it when you need the model to write or continue text. It is set up for mesh-llm. The card lists the license as other.
<div align="center" <a href="https://www.meshllm.cloud" <img src="https://meshllm.cloud/assets/images/jelly-logo-wordmark.png" alt="Mesh LLM" width="220" </a
Downloads · 30 days
55
19% of all-time downloads
All-time downloads
295
Public
Repo size
89.7 GB
Likes
0
Public
Click a slice to open those files.
.gguf89.7 GB · 100%
From the Hugging Face model README
Composite GGUF artifact combining the unsloth UD-Q4_K_XL quantization of NVIDIA Nemotron-3-Super-120B-A12B with NVIDIA's official MTPv2 multi-token prediction head, spliced in as a native NextN draft block for speculative decoding with Mesh LLM.
| Property | Value |
|---|---|
| Source target | unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF (UD-Q4_K_XL, 3 shards) |
| Source draft | nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2 (BF16 MTP head) |
| Base model | nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 |
| Family | Nemotron 3 (hybrid SSM + latent MoE) |
| GGUF architecture | nemotron_h_moe |
| Quantization | UD-Q4_K_XL (target) + BF16 (draft block) |
| Layers | 88 target + 1 MTP NextN block (blk.88) |
| MTP metadata | nemotron_h_moe.nextn_predict_layers=1 |
| Context length | 1,048,576 |
| Artifact size | ~84 GiB across 3 shards |
| Artifact | Size | Contents |
|---|---|---|
...-00001-of-00003.gguf | 7 MB | Metadata shard (rewritten: block_count 88→89, nextn_predict_layers=1) |
...-00002-of-00003.gguf | 46.5 GiB | Middle shard, byte-identical to the source |
...-00003-of-00003.gguf | 37.0 GiB | Last shard recomposed with the BF16 MTP draft tensors appended |
The target's quantized shards are kept untouched wherever possible; only shard 1
(metadata) and shard 3 (draft append) are rewritten. NVIDIA's BF16 MTPv2 head is
converted with the pinned llama.cpp converter (convert_hf_to_gguf.py --mtp),
which folds the two MTP layers into a single blk.88 NextN block, then spliced
past the target tensors with skippy-quantize compose-mtp (byte-copied
verbatim).
skippy-quantize compose-mtp + validate-mtp-attachbbd84c238311562f968bbbcaf3a8b2727a6ac642 (PR #1439)validate-mtp-attach passed on the published composite in the build job: the
3-part model loads and the MTP draft attaches natively (parts=3, layers=89,
mtp_layers=1, session created).
# Download the composite (all 3 shards).
hf download meshllm/NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL-MTPv2-GGUF \
--include '*-MTPv2-0000*-of-00003.gguf'
# Run locally with Mesh LLM (split serving across your machines).
mesh-llm serve --model "meshllm/NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL-MTPv2-GGUF"
# Check the local OpenAI-compatible model list.
curl -s http://localhost:3131/v1/models
MTP (multi-token prediction) lets the model produce a next token and an extra predicted token in the same decode step. The runtime uses the extra token as a speculative candidate and verifies it against the target instead of waiting for a full model pass per token — higher throughput at the same quality, since every accepted candidate is verified by the target itself.
This artifact derives from NVIDIA's Nemotron-3-Super-120B-A12B weights and unsloth's UD-Q4_K_XL quantization of them. License terms follow the source repositories — see nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 and unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF. For architecture details, chat template guidance, and sampling recommendations, see those model cards.