matryoshka-hypencoder

The Matryoshka Hypencoder

The Matryoshka Hypencoder is a novel neural information retrieval model that combines the expressive power of query-specific relevance functions (Hypencoders) with the computational flexibility of Matryoshka Representation Learning (MRL).

This model generates a query-specific scoring network (a "Q-Net") whose parameters are structured as nested representations. This allows a single model to generate Q-Nets of varying widths (e.g., hidden dimensions of 128, 256, 512, or 768), enabling a dynamic trade-off between retrieval effectiveness and inference latency without needing to train or deploy multiple separate models.

This model was developed as part of a Master's dissertation extending the original Hypencoder architecture by Killingback et al.

Model Details

• Architecture: Hypencoder Dual Encoder (Frozen BERT-base backbones + Trainable Matryoshka Hyper-head)
• Supported Matryoshka Dimensions: [128, 256, 512, 768]
• Training Data: MS MARCO Passage Ranking (using teacher scores for knowledge distillation)
• Loss Function: Averaged, unweighted Matryoshka Margin-MSE
• Language: English

Performance & Efficiency

Brief overview of the results

The Matryoshka Hypencoder exhibits the property of graceful degradation. Truncating the Q-Net size drastically reduces inference latency while maintaining highly competitive retrieval performance.

In-Domain Performance (nDCG@10) & Efficiency:

*Speedup refers to the reduction in document scoring time over a large corpus compared to the full 768-dim Q-Net.

---

Detailed Results:

In-domain performance comparison:

(MH d refers to our Matryoshka Hypencoder at a Q-Net hidden dimension of d. Hyp-Fr is our frozen-encoder model. † indicates a statistically significant difference (p < 0.05) compared to the Original Hypencoder baseline, and * shows significance compared to MH d=768.)

---

Zero-shot performance (nDCG@10) on out-of-domain BEIR datasets:

(TC is TREC-COVID, FQA is FiQA, NF is NFCorpus, DB is DBPedia, and WT is Webis-Touché. † indicates a statistically significant difference (p < 0.05) compared to the Original Hypencoder baseline, and * shows significance compared to MH d=768.)

---

Average document scoring throughput:

(Documents scored per second. The speed up compared to the full-size Q-Net (d=768) is given in parentheses.)

---

How to Use this Model

Because this model uses a custom architecture, it requires the custom hypencoder-cb codebase to run. It cannot be used with standard transformers pipelines out-of-the-box.

1. Installation

First, clone the official implementation repository and install the dependencies:

git clone <URL_TO_YOUR_GITHUB_REPO>  # Replace with your actual GitHub repo URL
cd hypencoder-paper
pip install -r requirements.txt

2. Loading the Model in Python

You can load the model directly from the Hugging Face Hub using the custom classes provided in the repository:

import torch
from hypencoder_cb.modeling.hypencoder import HypencoderDualEncoder
from transformers import AutoTokenizer

model_id = "majdalkawaas/matryoshka-hypencoder"
device = "cuda" if torch.cuda.is_available() else "cpu"

# Load the custom model architecture and weights
model = HypencoderDualEncoder.from_pretrained(model_id).to(device).eval()
tokenizer = AutoTokenizer.from_pretrained(model_id)

print(f"Model successfully loaded. Supported Q-Net sizes: {model.config.loss_kwargs[0]['matryoshka_dims']}")

3. Generating a Matryoshka Q-Net

To generate a Q-Net for a specific query and a specific Matryoshka dimension (e.g., 256):

from hypencoder_cb.modeling.q_net import MatryoshkaQNetFactory
from hypencoder_cb.modeling.similarity_and_losses import _truncate_parameters

query = "What is a hypernetwork?"
target_dimension = 256

# Tokenize the query
inputs = tokenizer(query, return_tensors="pt", truncation=True).to(device)

with torch.no_grad():
    # 1. Generate the full-size parameters from the Hyper-head
    query_output = model.query_encoder(**inputs)
    
    # 2. Build the specific Q-Net using the Factory
    factory = MatryoshkaQNetFactory(model.query_encoder.weight_to_model_converter)
    q_nets = factory.build(
        weight_matrices=query_output.generated_matrices,
        bias_vectors=query_output.generated_vectors,
        matryoshka_dims=[target_dimension],
        is_training=False
    )
    
    # This is your highly efficient, query-specific scoring function
    my_q_net = q_nets[target_dimension]

# You can now use `my_q_net` to score pre-computed document embeddings!

Citation

If you use this model in your research, please cite the original Hypencoder paper and this dissertation work.

@inproceedings{Alkawaas_2026,
   title={The Matryoshka Hypencoder},
   url={http://dx.doi.org/10.1145/3805712.3809980},
   DOI={10.1145/3805712.3809980},
   booktitle={Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval},
   publisher={ACM},
   author={Alkawaas, Majd and MacAvaney, Sean},
   year={2026},
   month=July, pages={3574–3579} }