Downloads · 30 days
34
4% of all-time downloads
Myric/Nemotron-3-Super-120B-A12B-APEX-GGUF
Nemotron-3-Super-120B-A12B-APEX-GGUF is a text generation model from Myric. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as other.
APEX (measured/structural per-tensor bit allocation) GGUF quants of nvidia/NVIDIA-Nemotron-3-Super-120B-A12B — 120B total / ~12B active. Architecturally the most unusual model this pipeline has handled: a hybrid backb…
Downloads · 30 days
34
4% of all-time downloads
All-time downloads
800
Public
Repo size
154 GB
Likes
0
Public
Click a slice to open those files.
.gguf154 GB · 100%
From the Hugging Face model README
APEX (measured/structural per-tensor bit allocation) GGUF quants of nvidia/NVIDIA-Nemotron-3-Super-120B-A12B — 120B total / ~12B active. Architecturally the most unusual model this pipeline has handled: a hybrid backbone where each of 88 layers is exactly one of three mixer types (never a combination) — 8 pure-Attention layers, 40 pure-Mamba2 (linear-attention/SSM) layers, and 40 MoE-FFN layers whose experts route through a shared low-rank latent bottleneck (NVIDIA's "LatentMoE") rather than a standard SwiGLU gate.
Status: two tiers available in this upload (i-quality, i-compact — see note below on i-mini). Real-generation coherence + chained tool-calling (dependent 3-turn chain + a distractor tool that must never be called) both pass 3/3 — see GATE.md. No PPL-ratio: the true bf16 reference is ~240GB and doesn't fit resident on the build hardware to run a baseline pass.
--ctx-size explicitly — do not run this model with defaultsNemotron-3-Super was trained with up to a 1,048,576-token (1M) context window. If you launch
llama-cli/llama-server without an explicit --ctx-size, llama.cpp defaults the KV cache to
the model's own trained context length, not a small sane default — an attempt to allocate a KV
cache sized for a million tokens, which can consume very large amounts of memory and stall or
crash a machine with limited RAM/VRAM.
Always pass --ctx-size sized to what you actually need, e.g. --ctx-size 8192 for typical
chat/tool-use. Only reach for six-figure-plus context sizes if you have the RAM/VRAM to back it.
| size | routed-expert precision | file size | bits/weight | coherent | tool calls |
|---|---|---|---|---|---|
| i-quality | Q6_K (edge/near bands), Q4_K (middle) | 85.78 GB | 5.10 | yes | 3/3 |
| i-compact | Q5_K/Q4_K (edge/near), Q3_K (middle) | 67.82 GB | 4.02 | yes | 3/3 |
i-mini isn't part of this upload round. Its planned routed-expert middle band targeted
IQ2_XXS, but moe_intermediate_size=2688 isn't divisible by 256, so llama-quantize can't apply
that i-quant type there and silently falls back to a higher-precision type for those tensors
(WARNING: 40 of 763 tensor(s) required fallback quantization) — landing at 3.81 bpw / 53.6 GB
instead of the ~2.8 bpw / 42 GB originally targeted. Worth remembering for any future recipe on
this architecture.
Attention, the two Mamba2 projections (ssm_in/ssm_out), and the shared latent bottleneck stay
at higher precision (Q6_K–Q8_0) in every tier; all state-space scalars/norms (ssm_conv1d,
ssm_a, ssm_d, ssm_dt.bias, ssm_norm.weight) and the router stay F32, same as the reference
community quant.
Built without ever downloading the model's raw safetensors at inference precision twice over. The
imatrix was computed by a new band-serialized generator
(serialized_gen_nemotron.py/imatrix_sources.py's nemotron_h_moe arch adapter) that drives the
real transformers NemotronHBlock module — materializing one layer's weights at a time from a
full-precision BF16 GGUF and hooking its real forward pass — rather than hand-reimplementing the
Mamba2 selective-scan recurrence, which would be far more likely to have a subtle bug than driving
the library's own tested module. Three GGUF-side tensor transforms (A_log = log(-ssm_a),
D/ssm_norm.weight reshapes, conv1d.weight unsqueeze) were verified against llama.cpp's own
conversion source before trusting them, not guessed.
Bit allocation is structural (role/depth-band priors) — this arch doesn't have a measured- sensitivity map yet (would need a memory-bounded redesign of that mode, deferred for now).
Imatrix computed over a large, deliberately diverse local corpus (~2MB: encyclopedic/wikitext prose, source code in several languages, classic literature, and technical/scientific text), 126 chunks × 512 tokens. Of the 40,960 total (expert × up/down-tensor) rows across all MoE layers, only 8 never saw a routed token across the whole corpus — strong coverage of the full 512-expert population.
Unofficial community quantization; not affiliated with or endorsed by NVIDIA. Use of this model is subject to the NVIDIA Nemotron Open Model License linked above.