Downloads · 30 days
54
2% of all-time downloads
FaisalFehad/Nemotron-3-Super-120B-A12B-MLX-6bit
Nemotron-3-Super-120B-A12B-MLX-6bit is a text generation model from FaisalFehad. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as other.
MLX quantization of nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 for Apple Silicon.
Downloads · 30 days
54
2% of all-time downloads
All-time downloads
2.2K
Public
Parameters
121B
98.2 GB on disk
Likes
2
Public
Click a slice to open those files.
.safetensors98.1 GB · 100%
How the weights are stored.
U32121B · 100%
From the Hugging Face model README
MLX quantization of nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 for Apple Silicon.
| Detail | Value |
|---|---|
| Architecture | Hybrid Mamba-2 + Transformer Attention + Latent MoE |
| Total Parameters | 120B |
| Active Parameters | 12B per token |
| Context Length | 1M tokens (262,144 default) |
| Experts | 512 routed, 22 active per token, 1 shared |
| Quantization | 6-bit affine (6.507 BPW), group size 64 |
| Disk Size | ~92 GB |
| Peak Memory | ~98.4 GB |
mlx-lm >= 0.31.2 (install from git main for Latent MoE support)pip install git+https://github.com/ml-explore/mlx-lm.git
mlx_lm.generate \
--model FF-01/Nemotron-3-Super-120B-A12B-MLX-6bit \
--prompt "Hello!" \
--max-tokens 256
from mlx_lm import load, generate
model, tokenizer = load("FF-01/Nemotron-3-Super-120B-A12B-MLX-6bit")
response = generate(model, tokenizer, prompt="Hello!", max_tokens=256)
print(response)
This model is compatible with LM Studio on Apple Silicon. Search for FF-01/Nemotron-3-Super-120B-A12B-MLX-6bit in the model browser and download directly.
Tested on M5 Pro Max (128GB):
| Metric | Value |
|---|---|
| Generation Speed | ~43.6 tok/s |
| Peak Memory | 98.4 GB |
Nemotron-H is a hybrid architecture combining three components:
The layer pattern alternates between Mamba (M) and attention with MoE (E) blocks across 88 layers. This hybrid design achieves strong performance with only 12B active parameters per token despite having 120B total.
This is a reasoning model that outputs chain-of-thought before the final answer. The model uses <think> and </think> tags to delineate reasoning.