ettin-glint-kd

Multi-Vector Encoder

This is a Multi-Vector Encoder model trained on the parquet dataset using the sentence-transformers library. It maps inputs to sequences of 128-dimensional token-level vectors and scores them with late interaction (MaxSim), useful for semantic search with late interaction.

Model Details

Model Description

• Model Type: Multi-Vector Encoder

<!-- - **Base model:** [Unknown](https://huggingface.co/unknown) -->

• Maximum Sequence Length: 300 tokens
  • Maximum Query Length: 32 tokens
  • Maximum Document Length: 300 tokens
• Output Dimensionality: 128 dimensions
• Similarity Function: MaxSim
• Supported Modality: Text
• Training Dataset:
  • parquet

<!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Multi-Vector Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Multi-Vector Encoders on Hugging Face

Full Model Architecture

MultiVectorEncoder(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'query_length': 32, 'document_length': 300, 'architecture': 'ModernBertModel'})
  (1): Dense({'in_features': 256, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings'})
  (2): MultiVectorMask({'skiplist_words': [], 'skiplist_tasks': ['document'], 'keep_only_token_ids': None})
  (3): Normalize({'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings'})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import MultiVectorEncoder

# Download from the 🤗 Hub
model = MultiVectorEncoder("multi_vector_encoder_model_id")
# Run inference: each input becomes a sequence of per-token vectors (variable length).
queries = [
    'What actor stared in a film directed by Mark L. Lester?',
]
documents = [
    'Commando is a 1985 American action film directed by Mark L. Lester, and starring Arnold Schwarzenegger and Rae Dawn Chong. The film was released in the United States on October 4, 1985.',
    'Drew Snyder is an American actor of film and television. He is best known for his roles in films such as "Commando, Firestarter," and "Cruel Intentions", a well as numerous guest and recurring roles in several well-known television series including "American Horror Story", "NYPD Blue", and "Life Goes On".',
    'Mark L. Lester (born November 26, 1946) is an American film director, screenwriter, and producer. He is best known as a prolific director of cult films including the disco musical "Roller Boogie", the vigilante thriller film "Class of 1984", the Stephen King-adaptation "Firestarter" (1984), the Arnold Schwarzenegger action film "Commando" (1985), and the action-comedy "Armed and Dangerous" (1986), starring John Candy, Eugene Levy, and Meg Ryan.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings[0].shape, document_embeddings[0].shape)
# (16, 128) (47, 128)

# Get the MaxSim similarity scores
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[11.4864,  5.9646, 12.2358]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

parquet

• Dataset: parquet
• Size: 1,046,009 training samples
• Columns: <code>query</code>, <code>document_1</code>, <code>document_2</code>, <code>document_3</code>, <code>document_4</code>, <code>document_5</code>, <code>document_6</code>, <code>document_7</code>, <code>document_8</code>, <code>document_9</code>, <code>document_10</code>, <code>document_11</code>, <code>document_12</code>, <code>document_13</code>, <code>document_14</code>, <code>document_15</code>, <code>document_16</code>, <code>document_17</code>, <code>document_18</code>, <code>document_19</code>, <code>document_20</code>, <code>document_21</code>, <code>document_22</code>, <code>document_23</code>, <code>document_24</code>, <code>document_25</code>, <code>document_26</code>, <code>document_27</code>, <code>document_28</code>, <code>document_29</code>, <code>document_30</code>, <code>document_31</code>, <code>document_32</code>, and <code>scores</code>
• Approximate statistics based on the first 100 samples:

  	query	document_1	document_2	document_3	document_4	document_5	document_6	document_7	document_8	document_9	document_10	document_11	document_12	document_13	document_14	document_15	document_16	document_17	document_18	document_19	document_20	document_21	document_22	document_23	document_24	document_25	document_26	document_27	document_28	document_29	document_30	document_31	document_32	scores
  type	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	string	list
  modality	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text	text
  details	<ul><li>min: 7 tokens</li><li>mean: 14.26 tokens</li><li>max: 24 tokens</li></ul>	<ul><li>min: 15 tokens</li><li>mean: 178.02 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 31 tokens</li><li>mean: 174.67 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 18 tokens</li><li>mean: 177.12 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 45 tokens</li><li>mean: 181.39 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 26 tokens</li><li>mean: 168.93 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 26 tokens</li><li>mean: 173.92 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 27 tokens</li><li>mean: 170.85 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 24 tokens</li><li>mean: 176.45 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 38 tokens</li><li>mean: 186.77 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 21 tokens</li><li>mean: 167.45 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 12 tokens</li><li>mean: 171.02 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 21 tokens</li><li>mean: 187.41 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 24 tokens</li><li>mean: 175.52 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 32 tokens</li><li>mean: 164.27 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 52 tokens</li><li>mean: 192.5 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 19 tokens</li><li>mean: 178.11 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 18 tokens</li><li>mean: 169.45 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 34 tokens</li><li>mean: 172.09 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 22 tokens</li><li>mean: 172.31 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 27 tokens</li><li>mean: 177.35 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 24 tokens</li><li>mean: 171.45 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 15 tokens</li><li>mean: 166.03 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 19 tokens</li><li>mean: 172.17 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 29 tokens</li><li>mean: 165.82 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 36 tokens</li><li>mean: 185.49 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 28 tokens</li><li>mean: 183.41 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 29 tokens</li><li>mean: 175.08 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 31 tokens</li><li>mean: 160.34 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 22 tokens</li><li>mean: 172.46 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 25 tokens</li><li>mean: 166.98 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 27 tokens</li><li>mean: 169.66 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>min: 28 tokens</li><li>mean: 161.57 tokens</li><li>max: 300 tokens</li></ul>	<ul><li>size: 32 elements</li></ul>

• Samples:

  query	document_1	document_2	document_3	document_4	document_5	document_6	document_7	document_8	document_9	document_10	document_11	document_12	document_13	document_14	document_15	document_16	document_17	document_18	document_19	document_20	document_21	document_22	document_23	document_24	document_25	document_26	document_27	document_28	document_29	document_30	document_31	document_32	scores

The model card is truncated. Read the rest on Hugging Face.