Nemotron-3-Nano-30B-A3B-context-mlx-4bit

Nemotron-3-Nano-30B-A3B (context tower) — MLX 4-bit

The autoregressive context tower of nvidia/Nemotron-Labs-TwoTower-30B-A3B-Base-BF16, extracted and converted to MLX (4-bit). This is the frozen base model (NemotronH hybrid: 52 layers = 23 Mamba-2, 6 attention, 23 MoE; 128 experts, 6 active + 1 shared; ~30B params, ~3B active). It runs directly in stock mlx-lm as an ordinary text model — no custom code.

This is the AR backbone only. For the actual two-tower diffusion behavior, use the Nemotron-Labs-TwoTower-30B-A3B-mlx-* repos below.

📦 Usage guide, examples & benchmarks: https://github.com/PipeNetwork/nemotron-twotower-mlx

Usage

pip install mlx-lm
mlx_lm.generate --model pipenetwork/Nemotron-3-Nano-30B-A3B-context-mlx-4bit \
  --prompt "The capital of France is" --max-tokens 128

from mlx_lm import load, generate
model, tok = load("pipenetwork/Nemotron-3-Nano-30B-A3B-context-mlx-4bit")
print(generate(model, tok, prompt="The key idea behind Mamba is", max_tokens=128))

Model family

Validation

The MLX conversion was checked against NVIDIA's reference implementation running on an NVIDIA GB10 (CUDA). Greedy decoding matched token-for-token: 120/120 tokens (100%) and 5/5 top-1 across the test prompts — e.g. both produce "George Washington. He was elected in 1789 and served two terms until 1797." (The AR/context tower is exercised directly; it is the shared backbone the diffusion denoiser also uses.)

Benchmarks

Measured on Apple M3 Ultra (512 GB unified memory), MLX 0.31 — steady-state (post-warmup). Peak RAM is the unified-memory high-water mark during generation.

AR / context tower — 128-token single-stream generation:

TwoTower diffusion — 64 new tokens, block size 16, ≤16 steps/block:

Diffusion runs steps_per_block denoiser passes per block, so it is slower per token than the AR tower — lower --steps-per-block trades quality for speed. Higher-precision builds tend to converge in fewer denoiser evaluations, but each pass moves more memory.

License

Governed by the NVIDIA Open Model License of the base model.