Downloads · 30 days
59
3% of all-time downloads
batiai/Nemotron-3-Nano-30B-A3B-GGUF
Nemotron-3-Nano-30B-A3B-GGUF is a text generation model from batiai. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as other.
<p align="center" <a href="https://flow.bati.ai"<img src="https://img.shields.io/badge/BatiFlow-macOS%20AI%20Automation-blue?style=for-the-badge&logo=apple" alt="BatiFlow"</a <a href="https://ollama.com/batiai/nemotro…
Downloads · 30 days
59
3% of all-time downloads
All-time downloads
2.1K
Public
Repo size
61.9 GB
Likes
0
Public
Click a slice to open those files.
.gguf61.9 GB · 100%
From the Hugging Face model README
Quantizations of NVIDIA Nemotron 3 Nano 30B-A3B (NemotronH MoE, hybrid Mamba+Attention) for on-device AI on Mac. Built and verified by BatiAI for BatiFlow.
ollama pull batiai/nemotron3-nano:iq4
| Quant | Size | VRAM target | Recommended For |
|---|---|---|---|
| IQ3_XXS | 17GB | ~20GB | 24–32GB Mac |
| IQ4_XS | 17GB | ~20GB | 32GB Mac (recommended) |
| Q5_K_M | 25GB | ~28GB | 36GB+ Mac (highest quality) |
| Your Mac RAM | IQ3_XXS (17GB) | IQ4_XS (17GB) | Q5_K_M (25GB) |
|---|---|---|---|
| 16GB | ⚠️ Heavy swap | ⚠️ Heavy swap | ❌ |
| 24GB | ✅ | ✅ | ❌ |
| 32GB | ✅ Fast | ✅ Recommended | ⚠️ Tight |
| 36GB+ | ✅ | ✅ | ✅ Best quality |
| 48GB+ | ✅ | ✅ | ✅ Headroom |
| Your Mac | Best Model | Notes |
|---|---|---|
| 16GB | batiai/gemma4-e4b:q4 | Fast, lightweight |
| 24GB | batiai/gemma4-26b:iq4 or batiai/nemotron3-nano:iq3 | Reasoning + tools |
| 32GB | batiai/nemotron3-nano:iq4 | Hybrid MoE, agentic |
| 36GB | batiai/qwen3.5-35b:iq4 | Alibaba MoE |
| 48GB | batiai/gemma4-31b:iq4 or batiai/nemotron3-nano:q5 | High quality |
| 128GB | batiai/minimax-m2.7:iq3 (229B) | Frontier on laptop |
NemotronH is NVIDIA's hybrid architecture combining Mamba state-space layers with standard attention:
| BatiAI | Third-party (TheBloke, etc.) | |
|---|---|---|
| Source | Quantized from official NVIDIA weights | Re-quantized from other GGUFs |
| Tested on | Real Mac hardware | Often untested on consumer hardware |
| imatrix | ✅ Calibrated (200 chunks wikitext-2) | Standard or none |
| Tool Calling | ✅ Verified | Often untested |
| Korean | ✅ Validated | Not tested |
imatrix --chunks 200 calibratedBatiFlow — free, on-device AI automation for Mac. 5MB app, 100% local, unlimited. 57+ built-in tools for calendar, notes, reminders, files, email, browser, messaging.
Quantized from nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16. License: NVIDIA Open Model License.
| Machine | Quant | Cold start | Prompt eval | Token gen | Tested |
|---|---|---|---|---|---|
| MacBook Pro M4 Max 128GB | IQ3_XXS | 1.599s | 208.95 t/s | 86.15 t/s | 2026-05-03 |
| MacBook Pro M4 Max 128GB | IQ4_XS | 1.589s | 206.43 t/s | 88.77 t/s | 2026-05-03 |
| MacBook Pro M4 Max 128GB | Q5_K_M | 5.036s | 179.26 t/s | 75.82 t/s | 2026-05-03 |