elephant-rerank-v1-text-small

Elephant Rerank V1 Text Small

elephant-rerank-v1-text-small is the text reranker model in the Agentic Intelligence Lab Elephant Rerank V1 family.

This ModelScope release is maintained by agentic-intelligence-lab to make Elephant rerank models easier to download and deploy in mainland China. It mirrors and renames the upstream HuggingFace model llm-semantic-router/mmbert-rerank-32k-2d-matryoshka under a consistent Elephant model namespace.

Positioning

This model is a multilingual long-context cross-encoder reranker for retrieval pipelines, agent memory systems, and RAG applications.

Embedding models are usually used for fast candidate generation. A reranker is used after that stage to score query-document pairs with higher precision. elephant-rerank-v1-text-small is designed for the second stage: take a query and a set of candidate passages, then assign relevance scores for final ordering.

The model is especially useful when passages are longer than the 512-token window used by many rerankers, or when relevant information may appear late in a document.

Model at a glance

Why it fits agentic workloads

Agentic systems often retrieve many candidate memories, documents, tools, or execution traces before deciding what to use. The first retrieval stage needs to be fast; the final ordering stage needs to be precise. This reranker is designed for that final ordering stage.

Key advantages:

• Long-context pair scoring: score query-passage pairs with up to 32K tokens of context.
• Useful after vector retrieval: rerank candidates from Elephant embeddings or any other first-stage retriever.
• 2D Matryoshka flexibility: use different layer and dimension heads to trade quality for cost.
• Multilingual coverage: suitable for mixed-language retrieval and international corpora.
• Agent-friendly use cases: memory selection, tool ranking, evidence ordering, and long-document RAG.

Recommended use cases

Quick start on ModelScope

pip install modelscope transformers torch

This package contains the ModernBERT encoder weights plus the 2D Matryoshka classification heads. Loading the full reranker requires the custom reranker wrapper used by the upstream training/export code.

import torch
from modelscope import snapshot_download
from transformers import AutoTokenizer

# Use the reranker wrapper from the upstream training package.
# The wrapper is expected to load `model.safetensors`, `classification_heads.pt`,
# and `matryoshka_config.json` from the local model directory.
from train_rerank import Matryoshka2DReranker

repo_id = "agentic-intelligence-lab/elephant-rerank-v1-text-small"
local_dir = snapshot_download(repo_id)

model = Matryoshka2DReranker.from_pretrained(local_dir)
tokenizer = AutoTokenizer.from_pretrained(local_dir)

model.eval()
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)

pairs = [
    (
        "What is machine learning?",
        "Machine learning is a subset of AI that enables systems to learn from data.",
    ),
    (
        "What is machine learning?",
        "The weather is sunny today.",
    ),
]

scores = model.compute_score(pairs, tokenizer, normalize=True)
print(scores)

2D Matryoshka scoring

The model provides multiple layer and dimension heads. This allows one checkpoint to serve several quality/cost profiles.

# Full model: 22 layers, 768 dimensions
scores_full = model.compute_score(pairs, tokenizer, normalize=True)

# Balanced profile
scores_balanced = model.compute_score(
    pairs,
    tokenizer,
    layer_idx=11,
    dim_idx=256,
    normalize=True,
)

# Lower-cost profile
scores_fast = model.compute_score(
    pairs,
    tokenizer,
    layer_idx=6,
    dim_idx=128,
    normalize=True,
)

Reranking pipeline example

def rerank(query: str, passages: list[str], top_k: int = 10) -> list[tuple[str, float]]:
    pairs = [(query, passage) for passage in passages]
    scores = model.compute_score(pairs, tokenizer, normalize=True)
    ranked = sorted(zip(passages, scores), key=lambda item: item[1], reverse=True)
    return ranked[:top_k]

query = "How does photosynthesis work?"
passages = [
    "Photosynthesis is the process by which plants convert sunlight into energy.",
    "The stock market closed higher today.",
    "Plants use chlorophyll to absorb light during photosynthesis.",
    "Python is a popular programming language.",
]

results = rerank(query, passages, top_k=2)
print(results)

Evaluation snapshot

The long-document validation checks whether the reranker can still find relevant information when it appears late in a long passage. This is the main reason to use this model over short-window rerankers in long-context RAG and memory workflows.

Files

Lineage

This ModelScope package is published by agentic-intelligence-lab as part of the Elephant model release line. It mirrors the upstream HuggingFace model llm-semantic-router/mmbert-rerank-32k-2d-matryoshka and keeps the model artifacts unchanged except for the repository naming and model card presentation.

The model is built from llm-semantic-router/mmbert-32k-yarn, a ModernBERT-based multilingual encoder extended to 32K context with YaRN position interpolation.

Limitations

• This is a custom reranker export; the complete scoring path requires the upstream Matryoshka2DReranker wrapper or an equivalent implementation.
• Training data is primarily based on BGE-M3 style query-passage pairs, so specialized domains may benefit from fine-tuning.
• Although the model supports 32K tokens, very long query-passage pairs still increase compute and memory cost.
• Layer and dimension reduction trade quality for efficiency and should be validated for each production workload.
• For very short passages where latency is the only priority, a smaller short-window reranker may be faster.

Citation

@misc{elephant-rerank-v1-text-small,
  title={Elephant Rerank V1 Text Small},
  author={Agentic Intelligence Lab},
  year={2026},
  url={https://modelscope.cn/models/agentic-intelligence-lab/elephant-rerank-v1-text-small}
}

License

Apache 2.0