nomic-embed-text-v2-moe-GGUF

nomic-embed-text-v2-moe GGUF

Quantized GGUF versions of nomic-ai/nomic-embed-text-v2-moe for use with CrispEmbed.

Model Details

• Base model: nomic-ai/nomic-embed-text-v2-moe
• Architecture: NomicBERT with Mixture-of-Experts (MoE)
  • 12 layers, 768 hidden, 12 heads
  • 8 experts, top-2 routing, GELU activation
  • MoE on odd layers (1,3,5,7,9,11), dense GELU FFN on even layers
  • RoPE positional encoding, SentencePiece tokenizer (250k vocab)
• Output: 768-dim L2-normalized embeddings
• Context: 2048 tokens max

Available Quantizations

MoE expert weights (3D tensors) are quantized slice-by-slice, achieving full compression across all model weights including the 8-expert FFN layers.

Usage

CrispEmbed CLI

crispembed -m nomic-v2-moe-q8_0.gguf "search_query: What is a mixture of experts?"

CrispEmbed Python

from crispembed import CrispEmbed

model = CrispEmbed("nomic-v2-moe-q8_0.gguf")
embedding = model.encode("search_query: What is a mixture of experts?")

Text Prefixes

Following the original model's convention, prefix queries with search_query: and documents with search_document: for best retrieval performance.

Parity Verification

The F32 GGUF achieves cos = 1.000000 against HuggingFace Transformers across all test texts, verified with the CrispEmbed parity harness:

• Token IDs: exact match (SentencePiece)
• Weight tensors: bit-exact (148/148 tensors, max|delta| = 0.0)
• End-to-end embeddings: cos = 1.000000 on 4 test texts

Conversion

Converted using CrispEmbed's convert-bert-to-gguf.py:

python models/convert-bert-to-gguf.py \
    --model nomic-ai/nomic-embed-text-v2-moe \
    --output nomic-v2-moe.gguf --crisp

Quantized using crispembed-quantize:

crispembed-quantize nomic-v2-moe.gguf nomic-v2-moe-q8_0.gguf q8_0

License

Apache 2.0, following the base model license.

Provenance and EU AI Act Art. 53 note

• Upstream model: nomic-ai/nomic-embed-text-v2-moe — published by nomic-ai.
• Upstream licence: apache-2.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.