Downloads · 30 days
45
17% of all-time downloads
OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_4M
Nemotron-3-Nano-Omni-30B-A3B-JANG_4M is a image-text-to-text model from OsaurusAI. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as other.
Downloads · 30 days
45
17% of all-time downloads
All-time downloads
271
Public
Parameters
18.3B
43.6 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors21.8 GB · 100%
How the weights are stored.
U3216.9B · 92%
From the Hugging Face model README

21.8 GB · ~126.0 tok/s decode on M5 Max · 31B total / ~3B active · hybrid Mamba-2 + Attention + MoE · native MLX
Full multimodal (text + image + audio + video) port of NVIDIA's Nemotron-3-Nano-Omni-30B-A3B-Reasoning to Apple MLX — calibrated with a measured Hessian, an imatrix fit and AWQ, with the RADIO vision tower and the Parakeet speech encoder kept at fp16 so every modality actually works.
| Modality | Status |
|---|---|
| Text | ✅ reasoning on by default |
| Image (RADIO ViT, 512px tiles + thumbnail) | ✅ |
| Video (frame sampling + EVS pruning) | ✅ |
| Audio / ASR (Parakeet 24-layer Conformer) | ✅ |
| Tool calling (agentic) | ✅ XML <tool_call><function=…> |
JANG_4M is a mixed-precision allocation (4-bit, 8-bit), not a single width.
The bit map is chosen per tensor role, then calibrated:
| Component | Precision |
|---|---|
Routed experts switch_mlp.fc1 / fc2 (93% of params) | 4-bit / 4-bit |
Shared expert, attention q/k/v/o, lm_head | 8-bit |
Mamba-2 in_proj / out_proj, embeddings | 8-bit |
conv1d, A_log, D, dt_bias, router gate, all norms | fp16 |
| Vision tower, Parakeet encoder, both projectors | fp16 (never quantized) |
Group size 64. 164 quantized modules; 1106 tower tensors passed through untouched.
This model's expert MLP is unusually sensitive to low-bit quantization, so the routed experts carry a hard minimum width regardless of profile. A nominal "2-bit" build lands within ~7% of the 4-bit one and is measurably worse — not a 2-bit bundle in any useful sense, so we do not ship one.
Quantization is calibrated against measured activations from a domain-weighted corpus (coding, agentic/tool, reasoning, general, multilingual, long-context), not fitted to the weights alone:
QAT is not available for this model: NVIDIA's own quantized releases are
post-training (modelopt, MIXED_PRECISION), and NVFP4 codes carry no meaning
in an affine (scale, bias) layout.
Reasoning is ON by default (enable_thinking); turning it off prefills an
empty <think></think> block rather than omitting it.
| Mode | temperature | top_p | top_k | max_tokens | reasoning_budget | grace |
|---|---|---|---|---|---|---|
| Thinking (default) | 0.6 | 0.95 | — | 20480 | 16384 | 1024 |
| Instruct (non-thinking) | 0.2 | — | 1 | 1024 | — | — |
| ASR / transcription | 1.0 | — | 1 | — | — | — |
Stamped into both generation_config.json and
jang_config.chat.sampling_defaults. EOS is [2, 11] — dropping 11 means the
model never stops.
hf download OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_4M \
--local-dir Nemotron-3-Nano-Omni-30B-A3B-JANG_4M
Runs in Osaurus / vMLX, which carries the
native MLX NemotronHOmni stack (RADIO vision, Parakeet audio, EVS video). The
LLM half also loads directly with mlx_lm:
from mlx_lm import load, generate
model, tokenizer = load("Nemotron-3-Nano-Omni-30B-A3B-JANG_4M")
Quantized by Jinho Jang ([email protected]). Base model © NVIDIA, released under the NVIDIA Open Model License.