jina-colbert-v2

<br><br>

<p align="center"> <img src="https://huggingface.co/datasets/jinaai/documentation-images/resolve/main/logo.webp" alt="Jina AI: Your Search Foundation, Supercharged!" width="150px"> </p> <p align="center"> <b>Trained by <a href="https://jina.ai/"><b>Jina AI</b></a>.</b> </p> <p align="center"> <b>JinaColBERT V2: A General-Purpose Multilingual Late Interaction Retriever.</b> </p>

JinaColBERT V2 (jina-colbert-v2) is a new model based on the JinaColBERT V1 that expands on the capabilities and performance of the jina-colbert-v1-en model. Like the previous release, it has Jina AI’s 8192 token input context and the improved efficiency, performance, and explainability of token-level embeddings and late interaction.

This new release adds new functionality and performance improvements:

• Multilingual support for dozens of languages, with strong performance on major global languages.
• Matryoshka embeddings, which allow users to trade between efficiency and precision flexibly.
• Superior retrieval performance when compared to the English-only jina-colbert-v1-en.

JinaColBERT V2 offers three different versions for different embeddings dimensions: jinaai/jina-colbert-v2: 128 dimension embeddings jinaai/jina-colbert-v2-96: 96 dimension embeddings jinaai/jina-colbert-v2-64: 64 dimension embeddings

Usage

Installation

jina-colbert-v2 is trained with flash attention, so einops is required and flash_attn is recommended. Without flash_attn the model falls back to PyTorch's native attention implementation. Beyond that, pick whichever inference library you prefer.

pip install -U einops flash_attn

Sentence Transformers

This model can be used with Sentence Transformers as a multi-vector (ColBERT-style late interaction) retriever via the MultiVectorEncoder:

pip install "sentence-transformers>=6.0.0"

from sentence_transformers import MultiVectorEncoder

model = MultiVectorEncoder("jinaai/jina-colbert-v2", trust_remote_code=True)

query = "What does ColBERT do?"
documents = [
    "ColBERT is a novel ranking model that adapts deep LMs for efficient retrieval.",
    "Jina-ColBERT is a ColBERT-style model but based on JinaBERT so it can support both 8k context length, fast and accurate retrieval.",
]

query_embeddings = model.encode_query(query)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings[0].shape)
# (32, 128) (23, 128)

# MaxSim late-interaction scoring (higher is more relevant)
scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
# tensor([[23.2578, 21.5039]])

PyLate

pip install pylate

# Please refer to Pylate: https://github.com/lightonai/pylate for detailed usage
from pylate import indexes, models, retrieve

model = models.ColBERT(
    model_name_or_path="jinaai/jina-colbert-v2",
    query_prefix="[QueryMarker]",
    document_prefix="[DocumentMarker]",
    attend_to_expansion_tokens=True,
    trust_remote_code=True,
)

RAGatouille

pip install ragatouille

from ragatouille import RAGPretrainedModel

RAG = RAGPretrainedModel.from_pretrained("jinaai/jina-colbert-v2")
docs = [
    "ColBERT is a novel ranking model that adapts deep LMs for efficient retrieval.",
    "Jina-ColBERT is a ColBERT-style model but based on JinaBERT so it can support both 8k context length, fast and accurate retrieval.",
]
RAG.index(docs, index_name="demo")
query = "What does ColBERT do?"
results = RAG.search(query)

Stanford ColBERT

pip install colbert-ai

from colbert.infra import ColBERTConfig
from colbert.modeling.checkpoint import Checkpoint

ckpt = Checkpoint("jinaai/jina-colbert-v2", colbert_config=ColBERTConfig())
docs = [
    "ColBERT is a novel ranking model that adapts deep LMs for efficient retrieval.",
    "Jina-ColBERT is a ColBERT-style model but based on JinaBERT so it can support both 8k context length, fast and accurate retrieval.",
]
query_vectors = ckpt.queryFromText(docs, bsize=2)

Evaluation Results

Retrieval Benchmarks

BEIR

MS MARCO Passage Retrieval

Multilingual Benchmarks

MIRACLE

mMARCO

Matryoshka Representation Benchmarks

BEIR

MSMARCO

Other Models

Additionally, we provide the following embedding models, you can also use them for retrieval.

• jina-embeddings-v2-base-en: 137 million parameters.
• jina-embeddings-v2-base-zh: 161 million parameters Chinese-English bilingual model.
• jina-embeddings-v2-base-de: 161 million parameters German-English bilingual model.
• jina-embeddings-v2-base-es: 161 million parameters Spanish-English bilingual model.
• jina-reranker-v2: multilingual reranker model.
• jina-clip-v1: English multimodal (text-image) embedding model.

Contact

Join our Discord community and chat with other community members about ideas.

@inproceedings{xiao-etal-2024-jina,
    title = "{J}ina-{C}ol{BERT}-v2: A General-Purpose Multilingual Late Interaction Retriever",
    author = {Jha, Rohan  and
      Wang, Bo  and
      G{\"u}nther, Michael  and
      Mastrapas, Georgios  and
      Sturua, Saba  and
      Mohr, Isabelle  and
      Koukounas, Andreas  and
      Wang, Mohammad Kalim  and
      Wang, Nan  and
      Xiao, Han},
    editor = {S{\"a}lev{\"a}, Jonne  and
      Owodunni, Abraham},
    booktitle = "Proceedings of the Fourth Workshop on Multilingual Representation Learning (MRL 2024)",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.mrl-1.11/",
    doi = "10.18653/v1/2024.mrl-1.11",
    pages = "159--166",
    abstract = "Multi-vector dense models, such as ColBERT, have proven highly effective in information retrieval. ColBERT`s late interaction scoring approximates the joint query-document attention seen in cross-encoders while maintaining inference efficiency closer to traditional dense retrieval models, thanks to its bi-encoder architecture and recent optimizations in indexing and search. In this paper, we introduce a novel architecture and a training framework to support long context window and multilingual retrieval. Leveraging Matryoshka Representation Loss, we further demonstrate that the reducing the embedding dimensionality from 128 to 64 has insignificant impact on the model`s retrieval performance and cut storage requirements by up to 50{\%}. Our new model, Jina-ColBERT-v2, demonstrates strong performance across a range of English and multilingual retrieval tasks,"
}