Nemotron-Labs-3-Puzzle-75B-A9B-MLX-4bit-experts-6bit

Nemotron Labs 3 Puzzle 75B A9B, MLX mixed 4/6-bit

Mixed-precision MLX conversion of NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16, tested locally on an Apple M2 Max with 64 GB unified memory.

This is a community conversion, not an NVIDIA release.

Why this checkpoint exists

Puzzle 75B A9B has 75.3B total parameters and 9.3B active parameters per token. A uniform conversion is not the best fit for a 64 GB Mac because most parameters live in the routed expert bank, while dense paths and the output head are more sensitive.

This checkpoint applies precision by role:

The ordinary autoregressive backbone is included. Training-only MTP tensors were intentionally excluded.

Measured M2 Max results

Tested on an Apple M2 Max with 64 GB unified memory using MLX 0.32.0.

A separate 256-token Python-generation run measured 24.623 tok/s and 45.269 GB peak process memory. That run used a different prompt and should not be compared directly with the 30-run average.

The local checks are regression and instruction-following checks, not a standardized model-quality benchmark. See the benchmark methodology and raw metrics.

MLX support

This model needs the Nemotron Puzzle support implemented in:

• ml-explore/mlx-lm#1535
• sxuff/mlx-lm, feat/nemotron-h-puzzle-support

Until the upstream pull request is merged and released, install the feature branch:

git clone --branch feat/nemotron-h-puzzle-support \
  https://github.com/sxuff/mlx-lm.git
cd mlx-lm
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e .

Generate

mlx_lm.generate \
  --model tekosML/Nemotron-Labs-3-Puzzle-75B-A9B-MLX-4bit-experts-6bit \
  --trust-remote-code \
  --prompt "Write a Python function that returns the first n Fibonacci numbers." \
  --max-tokens 256 \
  --temp 1.0 \
  --top-p 0.95

Only use --trust-remote-code after inspecting the repository's configuration and modeling files. They were copied from the NVIDIA source checkpoint during conversion.

Conversion

The conversion script and exact heterogeneous quantization predicate are available at:

• sxuff/nemotron-puzzle-mlx
• scripts/convert_puzzle_mixed.py

Equivalent conversion command:

python scripts/convert_puzzle_mixed.py \
  /path/to/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16 \
  /path/to/output \
  --expert-bits 4

Recommended sampling settings used during validation:

temperature = 1.0
top_p = 0.95

Numerical compatibility work

The supporting MLX implementation preserves Puzzle's BF16 softplus(dt + dt_bias) boundary while retaining FP32 state-transition and recurrence arithmetic. First-layer SSM cosine similarity against NVIDIA's reference improved from 0.8832 to 0.999998 after correcting that boundary.

Puzzle-specific RMSNorm, router, projection-bias, and timestep behavior are gated to nemotron_h_puzzle, preserving existing ordinary Nemotron-H behavior.

Files and provenance

• Source weights: nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16
• Conversion runtime: MLX 0.32.0
• Format: sharded safetensors
• Weight shards: 9
• Total local files: 20
• File-content size: 45,104,700,954 bytes

License

Use and redistribution are governed by the original model's OpenMDW-1.1 license. The full license text and NVIDIA copyright notice are included in this repository.

The files configuration_nemotron_h.py and modeling_nemotron_h.py are separately licensed under Apache-2.0 as stated in the included license.

Original model copyright:

Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.

Users are responsible for reviewing and complying with the original model license.