Nemotron-3-Embed-1B-oQ2-MLX

Nemotron-3-Embed-1B-oQ2-MLX

<!-- polished-overview:start -->

Overview

Nemotron-3-Embed-1B-oQ2-MLX is an MLX-formatted checkpoint optimized for Apple silicon, published by groxaxo. It is intended for open-source evaluation, reproducible experimentation, and compatible local or hosted inference workflows. The wording below is deliberately limited to what can be verified from this repository's metadata and artifacts.

At a glance

What is included

• *.safetensors (1 file)
• config.json
• tokenizer.json
• tokenizer_config.json
• Additional configuration, tokenizer, processor, or shard files (14 visible artifacts total)

Quick start

MLX

Use an up-to-date MLX-compatible runtime on Apple silicon and point it at this repository:

mlx_lm.generate --model groxaxo/Nemotron-3-Embed-1B-oQ2-MLX --prompt "Write a concise technical summary."

Embedding and audio repositories may require the task-specific MLX package documented by the upstream project.

Compatibility and responsible use

• Use a runtime that explicitly supports this format, architecture, and modality.
• Keep configuration, tokenizer, processor, projection, and weight files from the same revision together.
• Review the source model card and license before redistribution or deployment.
• Hardware needs depend on parameter count, context length, cache precision, quantization, and concurrency.
• Report reproducible issues with the runtime version, hardware, launch command, and a minimal example.

Quantization or conversion changes numerical behavior, memory use, and throughput relative to the source checkpoint; validate quality on your own workload.

Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws.

<!-- polished-overview:end -->

An Apple-Silicon-native OMLX oQ2 conversion of nvidia/Nemotron-3-Embed-1B-BF16.

This derivative was produced locally on Apple Silicon with the same streaming, sensitivity-driven OMLX pipeline used for the oQ5 release. It uses a base 2-bit affine plan, group size 64, and bfloat16 working dtype. The calibration ran on the checkpoint's bidirectional encoder architecture rather than through a causal-LM wrapper. No weights were retrained.

The quantized safetensors payload is 427,993,279 bytes. OMLX promoted 23 sensitivity-selected projections to 5, 6-bit precision.

Embedding contract

• bidirectional attention (is_causal: false)
• average/mean pooling
• L2-normalized 2,048-dimensional output
• query: prefix for queries
• passage: prefix for documents

BF16 fidelity smoke test

The source and this oQ2 checkpoint were evaluated using a query, a matching passage, and an unrelated passage. All outputs were 2,048-dimensional and L2-normalized.

Quick start on Apple Silicon

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade mlx mlx-lm transformers huggingface_hub

hf download groxaxo/Nemotron-3-Embed-1B-oQ2-MLX --local-dir ./Nemotron-3-Embed-1B-oQ2-MLX
python ./Nemotron-3-Embed-1B-oQ2-MLX/embed_mlx.py \
  --model ./Nemotron-3-Embed-1B-oQ2-MLX \
  --query "Which city is known as the City of Sails?" \
  --passage "Auckland is widely known as the City of Sails."

The included script deliberately loads the bare Ministral3 backbone and applies non-causal attention. The regular MLX-LM generation path is causal and is not the correct embedding interface for this checkpoint.

Quantization details

See QUANTIZATION_REPORT.md for artifact SHA-256 values and the full local validation result.

License

This derivative inherits the OpenMDW-1.1 license from the source model. LICENSE, NOTICE, and THIRD_PARTY_NOTICES.md are included unchanged from NVIDIA's source repository.