Downloads · 30 days
2.3K
20% of all-time downloads
hoidhxd/NVIDIA-NemotronLabs-VoiceChat-11B-GGUF
NVIDIA-NemotronLabs-VoiceChat-11B-GGUF is a machine learning model from hoidhxd. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf.
Custom GGUF quantizations of NVIDIA/NemotronLabs-VoiceChat-11B.
Downloads · 30 days
2.3K
20% of all-time downloads
All-time downloads
11.6K
Public
Repo size
66.1 GB
Likes
14
Trending 1
Click a slice to open those files.
.gguf47.4 GB · 100%
From the Hugging Face model README
Custom GGUF quantizations of NVIDIA/NemotronLabs-VoiceChat-11B.
These files were converted from the original NVIDIA SafeTensors checkpoint to GGUF and then quantized using custom GGML-based C++ workers.
llama.cpp now runs this model, through the fork
sansamour/llama-voicechat.cpp.
Speech in, speech out, one duplex timeline, tool calls, on CPU or CUDA. No torch,
no NeMo.
The single-container GGUFs below cannot be loaded as they are: they carry NeMo
tensor names, no llama.cpp KV metadata and no tokenizer. Three converters in the
fork split one of them into the four files the runtime wants, and the result of
doing that to nemotron_voicechat_11b-Q4_0.gguf is in the llamacpp/ folder
of this repo:
| file | size | what |
|---|---|---|
llamacpp/nemotron_voicechat_11b-stt-llm-Q4_0.gguf | 4.67 GiB | the language model, a stock nemotron_h |
llamacpp/nemotron_voicechat_11b-stt-llm-Q4_0-function-head.gguf | 315 MiB | the turn-taking / tool-call head, which llama.cpp would reject inside the model file |
llamacpp/mmproj-voicechat-perception-Q4_0.gguf | 435 MiB | the causal FastConformer speech encoder, as an mtmd audio projector |
llamacpp/voicechat-tts-Q4_0.gguf | 686 MiB | the speech generator: gemma3 backbone, MoG head, 31 stage RVQ, codec decoder |
hf download hoidhxd/NVIDIA-NemotronLabs-VoiceChat-11B-GGUF --include "llamacpp/*" --local-dir .
llama-voicechat -m llamacpp/nemotron_voicechat_11b-stt-llm-Q4_0.gguf --mmproj llamacpp/mmproj-voicechat-perception-Q4_0.gguf --tts llamacpp/voicechat-tts-Q4_0.gguf --audio question.wav --tts-out answer.wav
Keep the four files in one folder; the function head is found by name next to the
model. Set VC_NO_BARGE=1 and VC_FORCE_BOS=1 for anything push-to-talk - the
model is duplex, so left alone it opens its own turn about a second into the clip
and answers only that first second.
Nothing was requantized on the way. A Q4_0 tensor in the source is copied out block for block and stays bit-identical.
The model answers in English only.
nvidia/NVIDIA-NemotronLabs-VoiceChat-11BThe original model and its configuration are available at:
https://huggingface.co/nvidia/NVIDIA-NemotronLabs-VoiceChat-11B
| File | Quantization | Size |
|---|---|---|
nemotron_voicechat_11b-Q8_0.gguf | Q8_0 | ~12.04 GB |
nemotron_voicechat_11b-Q4_0.gguf | Q4_0 | ~6.62 GB |
Actual files produced during conversion:
Output size: 12,035,312,704 bytes
Final size : 11.2088 GiB
Q8_0 tensors : 1369
Kept tensors : 263
F16 kept : 257
Non-F16 kept : 6
Output size: 6,618,150,208 bytes
Final size : 6.16363 GiB
Q4_0 tensors : 1369
F16 kept : 257
Non-F16 kept : 6
The conversion was performed entirely locally.
NVIDIA SafeTensors
|
v
model.safetensors
|
| custom Python converter
v
nemotron_voicechat_11b-f16.gguf
|
+----------------------------+
| |
| custom C++ GGML worker | custom C++ GGML worker
v v
Q8_0 Q4_0
| |
v v
nemotron_voicechat_11b- nemotron_voicechat_11b-
Q8_0.gguf Q4_0.gguf
The F16 intermediate GGUF contained:
1632 tensors
GGUF V3
The quantization workers preserve:
Only eligible F16 tensors are quantized.
The Q8_0 conversion uses a custom C++ worker based on GGML quantization routines.
The policy used for this build is:
F16 tensor
|
+-- compatible with Q8_0
| |
| +--> Q8_0
|
+-- incompatible
| |
| +--> keep F16
|
+-- non-F16
|
+--> keep original type
Result:
1369 tensors -> Q8_0
257 tensors -> F16
6 tensors -> original non-F16 type
The Q4_0 conversion uses a separate custom C++ worker.
Policy:
F16 tensor
|
+-- compatible with Q4_0
| |
| +--> Q4_0
|
+-- incompatible
| |
| +--> keep F16
|
+-- non-F16
|
+--> keep original type
Result:
1369 tensors -> Q4_0
257 tensors -> F16
6 tensors -> original non-F16 type
An unmodified llama.cpp binary cannot run these files. llama-quantize cannot
touch them either:
unknown model architecture: 'nemotron_voicechat'
That is expected: this is one container holding five models under NeMo tensor
names, and llama.cpp has no such architecture. The fork
sansamour/llama-voicechat.cpp
does not add one - it splits the container into pieces llama.cpp already
understands (a nemotron_h model, an mtmd audio projector, and two side GGUFs
read by the tool itself). See Running these files above;
the converted set is in llamacpp/.
Approximate target sizes:
~12 GB model file.
Suitable for testing on GPUs with approximately:
~6.6 GB model file.
Suitable for testing on GPUs with approximately:
Actual runtime VRAM usage will be higher than the GGUF file size because of:
Therefore the GGUF file size should NOT be interpreted as the exact VRAM requirement.
These are custom quantizations.
No claim is made here that Q8_0 or Q4_0 produces identical output quality to the original NVIDIA SafeTensors model.
Users should benchmark:
against the original model.
Please refer to the original NVIDIA repository for:
https://huggingface.co/nvidia/NVIDIA-NemotronLabs-VoiceChat-11B
The license and usage conditions of the original NVIDIA model apply.
Please consult the original model repository before using or redistributing these files:
https://huggingface.co/nvidia/NVIDIA-NemotronLabs-VoiceChat-11B
This repository is an unofficial community conversion.
It is not an official NVIDIA release.
The quantized GGUF files were produced independently from the original NVIDIA checkpoint and are provided for research, experimentation and local inference development.
Original model:
NVIDIA NemotronLabs VoiceChat 11B
Original repository:
https://huggingface.co/nvidia/NVIDIA-NemotronLabs-VoiceChat-11B
GGUF conversion and quantization:
hoidhxd
Repository:
https://huggingface.co/hoidhxd/NVIDIA-NemotronLabs-VoiceChat-11B-GGUF