Downloads · 30 days
19
3% of all-time downloads
sapidlabs/Nemotron-3-Nano-30B-A3B-REAP-64e
Nemotron-3-Nano-30B-A3B-REAP-64e is a text generation model from sapidlabs. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
A 50% expert-pruned version of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B, produced with REAP (Router-weighted Expert Activation Pruning). Half the routed experts in each MoE layer were removed, shrinking the model from ~5…
Downloads · 30 days
19
3% of all-time downloads
All-time downloads
697
Public
Parameters
16.9B
33.8 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors33.8 GB · 100%
How the weights are stored.
BF1616.9B · 100%
From the Hugging Face model README
A 50% expert-pruned version of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B,
produced with REAP (Router-weighted Expert Activation Pruning). Half the
routed experts in each MoE layer were removed, shrinking the model from
~59 GB to ~32 GB while keeping the same routing width.
| Base | This model | |
|---|---|---|
| Routed experts / MoE layer | 128 | 64 |
| Experts activated per token | 6 | 6 (unchanged) |
| Total params | ~31.6 B | ~31.6 B nominal (fewer experts stored) |
| Size (BF16) | ~59 GB | ~32 GB |
| Architecture | nemotron_h (hybrid Mamba-Transformer MoE) | same |
REAP is a one-shot, post-training
method (no retraining) that scores every expert by the mean of
router_weight × activation_norm over a calibration set, then removes the
least-salient experts. Because only a few experts fire per token, many are
redundant — pruning them reduces the model's memory footprint. Note that
experts-per-token is unchanged (6), so the active compute per token is the
same; the win is total size / memory, not decode speed.
compression_ratio 0.5 → 64 of 128 routed experts kept per MoE layer.theblackcat102/evol-codealpaca-v1 at 512-token context.nemotron_h includes Mamba layers. With trust_remote_code=False (native
transformers modeling), it runs on a pure-torch fallback; install
mamba-ssm + causal-conv1d for the
fast path.from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
repo = "sapidlabs/Nemotron-3-Nano-30B-A3B-REAP-64e" # this repo
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
repo, torch_dtype=torch.bfloat16, device_map="cuda"
).eval()
msgs = [{"role": "user", "content": "Write a Python function for the nth Fibonacci number."}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True).to("cuda")
out = model.generate(**inputs, max_new_tokens=256)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Inherits the base model's license (NVIDIA Open Model License). This is a derivative of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B.