granite-embedding-311m-multilingual-r2-GGUF

Granite Embedding 311M Multilingual R2 - GGUF

This is a GGUF-format conversion of the ibm-granite/granite-embedding-311m-multilingual-r2 embedding model.

🛡️ Why this repository exists (Transparency & Trust)

I intentionally created this GGUF conversion from scratch rather than relying on pre-packaged vendor binaries because I believe in full transparency, trust, and control over the model pipeline. When running local AI, you should know exactly how your weights were processed.

To ensure anyone can audit the conversion or generate alternative quantization formats (like q8_0 or q4_K_M), I have provided the exact, reproducible Google Colab conversion script at the bottom of this model card. You do not have to trust my binary—you can verify it or build it yourself.

📊 Model Specifications

This model uses the ModernBERT architecture, bringing alternating attention, GeGLU activations, and rotary position embeddings (RoPE) to deliver state-of-the-art retrieval.

• Parameters: 311 Million
• Context Window: 32,768 tokens (Expanded 64x from the R1 version)
• Embedding Dimension: 768 (Supports Matryoshka Representation Learning, allowing truncation down to 512, 384, 256, or 128 dimensions)
• Format: GGUF (f16 16-bit float)
• Storage Size: ~623 MB
• Languages Supported: 200+ languages, with explicitly enhanced training for 52 languages and programming code (Python, Go, Java, JavaScript, C++, SQL, etc.).
• License: Apache 2.0

🏆 Performance & Evaluation

The Granite 311M Multilingual R2 model is highly optimized for multi-lingual and cross-lingual text search, code retrieval, and long-document search.

• Multilingual MTEB Retrieval (18 tasks): 65.2 score (a +13 point jump over the previous 278M generation).
• Code Retrieval: State-of-the-art performance for an open model under 500M parameters.
• Throughput: Capable of ~1,828 documents per second at 512-token lengths.

💻 How to Use

1. Using llama-cpp-python

You can easily load this embedding model in Python to generate vector embeddings for your documents or queries.

pip install llama-cpp-python
Python
from llama_cpp import Llama

# Load the model (ensure you download the .gguf file first)
model = Llama(
    model_path="granite-311m-multilingual-r2-f16.gguf",
    embedding=True,
    n_ctx=32768 # Support for full context window
)

# Generate an embedding
text = "The quick brown fox jumps over the lazy dog."
embedding = model.embed(text)

print(f"Embedding dimension: {len(embedding[0])}")