nemotron-3-embed-1b-gguf

Nemotron-3-Embed-1B GGUF

GGUF conversions of NVIDIA Nemotron-3-Embed-1B-BF16, a multilingual text embedding model optimized for retrieval and semantic similarity, prepared for local inference with llama.cpp-compatible runtimes and Ollama.

The model produces 2,048-dimensional, L2-normalized embeddings via mean pooling. It is a Ministral-3-based pruned encoder (16 layers, 1.14B parameters) trained by NVIDIA and evaluated across 34 languages.

Quantizations

All quantizations share the same embedding space as the BF16 original. Q8_0 is the recommended default for retrieval quality with a small memory footprint; F16 is the highest-fidelity GGUF option; Q4_K_M is the smallest option for memory-constrained hosts.

Usage with Ollama

Create a model from the GGUF file of your choice:

ollama create nemotron-3-embed-1b -f Modelfile

With a Modelfile such as:

FROM ./nemotron-3-embed-1b-q8_0.gguf
PARAMETER NUM_CTX 4096

Then embed text:

curl http://localhost:11434/api/embed \
  -d '{"model": "nemotron-3-embed-1b", "input": ["query: What is retrieval-augmented generation?", "passage: Retrieval-augmented generation adds retrieved documents to a model prompt."]}'

Usage with llama.cpp

llama-server -m nemotron-3-embed-1b-q8_0.gguf --pooling mean --ctx-size 4096
curl http://localhost:8080/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model": "nemotron-3-embed-1b", "input": ["text to embed"]}'

Conversion details

• Source: nvidia/Nemotron-3-Embed-1B-BF16 (revision 0677b2025cbc37daf92d7b9c7a225de8cfbc5b0b)
• Toolchain: llama.cpp (see CONVERSION_REPORT.md for the exact build and commands)
• Path: BF16 Safetensors -> F16 GGUF -> Q8_0 / Q4_K_M
• Embedding dimension: 2048, L2-normalized, mean pooling
• Context length: 262,144 declared; validated at 4,096 tokens

License

The model weights are derived from NVIDIA Nemotron-3-Embed-1B-BF16 and are licensed under the OpenMDW License Agreement, version 1.1. See LICENSE, NOTICE, and THIRD_PARTY_NOTICES.md in this repository. This conversion is not an official NVIDIA release and is not affiliated with or endorsed by NVIDIA.