Downloads · 30 days
1.9K
49% of all-time downloads
engram-ae/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-GGUF
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-GGUF is a any-to-any model from engram-ae. Use it for the any-to-any task on the model card, and read the license before you ship it in a product. The card lists the license as other.
NVIDIA Nemotron 3 Nano Omni is a multimodal large language model that unifies video, audio, image, and text understanding to support enterprise-grade Q&A, summarization, transcription, and document intelligence workfl…
Downloads · 30 days
1.9K
49% of all-time downloads
All-time downloads
3.9K
Public
Repo size
265 GB
Likes
1
Public
Click a slice to open those files.
.gguf264 GB · 100%
From the Hugging Face model README
| Total parameters | 31B (Mamba2-Transformer hybrid MoE) |
| Active parameters | ~3B per token |
| Modalities (in) | Video, Audio, Image, Text; video with its soundtrack in one pass |
| Modality (out) | Text |
| Reasoning mode | On by default |
| Quantization | Q4_K_M (GGUF); projector F16 (audio tower F16/BF16) |
| Files | Nine quantizations (Q3_K_M, IQ3_M, IQ4_XS, MXFP4_MOE, Q4_K_M, Q5_K_M, Q6_K, Q8_0, BF16) plus the unified projector mmproj-omni-vision-audio-F16.gguf (2.8 GB); single-tower projectors kept for compatibility. Every quantization is verified on image, audio, and video with soundtrack: see TEST-MATRIX.md |
| Runtime | llama.cpp: audio and video require the llama.cpp-omni fork (feat/nemotron-video); image and text run on stock |
| Minimum hardware | Any system llama.cpp supports with ~28 GB free memory; verified on NVIDIA DGX Spark (GB10, unified memory) |
| Mode | temperature | top_p | max_tokens |
|---|---|---|---|
| Thinking mode | 0.6 | 0.95 | 20480 |
| Transcription / deterministic | 0.0 | : | 2048 |
NVIDIA Nemotron 3 Nano Omni is a multimodal large language model that unifies video, audio, image, and text understanding to support enterprise-grade Q&A, summarization, transcription, and document intelligence workflows. It extends the Nemotron Nano family with integrated video+speech comprehension, Graphical User Interface (GUI), Optical Character Recognition (OCR), and speech transcription capabilities. NVIDIA Nemotron 3 Nano Omni was developed by NVIDIA as part of the Nemotron model family. <br>
This repository provides a GGUF release of the model for llama.cpp, quantized to Q4_K_M, together with a unified multimodal projector: a single mmproj GGUF carrying both the C-RADIO v4-H vision encoder (image and video frames) and the Parakeet speech encoder (audio inputs). Both towers are packaged in the mixed-modality GGUF layout llama.cpp resolves per modality (clip.vision.projector_type = nemotron_v2_vl, clip.audio.projector_type = parakeet), so one server process accepts video, audio, image, and text without swapping projector files. A video that carries an audio track is consumed in one pass: the frames go to the vision tower and the soundtrack is demuxed and fed to the audio tower in the same request, so the model answers about what it sees and what it hears from a single file. <br>
This model is available for commercial use. <br>
Governing Terms: Use of this model is governed by the NVIDIA Open Model Agreement <br>
Global <br>
This release is intended for users who require the Nemotron 3 Nano Omni multimodal capabilities: video and speech analysis, document intelligence (OCR, charts, long documents), speech transcription: on hardware served by llama.cpp, including single-GPU and unified-memory systems where the PyTorch serving stacks are not practical. <br>
Hugging Face 08/22/2026 <br>
Architecture Type: Mamba2-Transformer Hybrid Mixture of Experts (MoE) <br>
Network Architecture:
Changes from the source checkpoint: <br>
Input Type(s): Video, Audio, Image, Text <br>
Input Format(s): <br>
Output Type(s): Text <br>
Other Properties Related to Output: <br>
Runtime Engine(s):
feat/nemotron-video) <br>Supported Hardware Microarchitecture Compatibility: <br>
Preferred/Supported Operating System(s):
git clone https://github.com/VincentKaufmann/llama.cpp-omni
cd llama.cpp-omni && git checkout feat/nemotron-video
cmake -B build -DGGML_CUDA=ON && cmake --build build -j --target llama-mtmd-cli llama-server
# image
./build/bin/llama-mtmd-cli -m Nemotron-3-Nano-Omni-30B-A3B-Reasoning-Q4_K_M.gguf \
--mmproj mmproj-omni-vision-audio-F16.gguf --image photo.jpg -p "Describe this image."
# audio
./build/bin/llama-mtmd-cli -m Nemotron-3-Nano-Omni-30B-A3B-Reasoning-Q4_K_M.gguf \
--mmproj mmproj-omni-vision-audio-F16.gguf --audio speech.wav -p "Transcribe this audio."
# video: a soundtrack, when present, is heard in the same pass
./build/bin/llama-mtmd-cli -m Nemotron-3-Nano-Omni-30B-A3B-Reasoning-Q4_K_M.gguf \
--mmproj mmproj-omni-vision-audio-F16.gguf --video clip.mp4 \
-p "What is shown, and what does the narrator say?"
# server: one process, all modalities on /v1/chat/completions
./build/bin/llama-server -m Nemotron-3-Nano-Omni-30B-A3B-Reasoning-Q4_K_M.gguf \
--mmproj mmproj-omni-vision-audio-F16.gguf
The next release rebases this work onto current llama.cpp master and adopts the upstream audio projector layout. After that, image, text and audio run on stock llama.cpp with these files; the fork remains required only for video and one-pass audio+video.
NVIDIA believes Trustworthy AI is a shared responsibility. Please refer to the source model's model card for the training data, evaluation, bias, and safety documentation that applies to these weights. This repository changes the storage format and serving runtime only; it does not alter model behavior beyond the effects of Q4_K_M quantization. <br>