colbert-bert-mini

PyLate model based on prajjwal1/bert-mini

This is a PyLate model finetuned from prajjwal1/bert-mini on the msmarco-train dataset. It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.

Model Details

Model Description

• Model Type: PyLate model
• Base model: prajjwal1/bert-mini <!-- at revision 5e123abc2480f0c4b4cac186d3b3f09299c258fc -->
• Document Length: 256 tokens
• Query Length: 32 tokens
• Output Dimensionality: 128 tokens
• Similarity Function: MaxSim
• Training Dataset:
  • msmarco-train

<!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: PyLate Documentation
• Repository: PyLate on GitHub
• Hugging Face: PyLate models on Hugging Face

Full Model Architecture

ColBERT(
  (0): Transformer({'max_seq_length': 255, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Dense({'in_features': 256, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
)

Usage

First install the PyLate library:

pip install -U pylate

Retrieval

PyLate provides a streamlined interface to index and retrieve documents using ColBERT models. The index leverages the Voyager HNSW index to efficiently handle document embeddings and enable fast retrieval.

Indexing documents

First, load the ColBERT model and initialize the Voyager index, then encode and index your documents:

from pylate import indexes, models, retrieve

# Step 1: Load the ColBERT model
model = models.ColBERT(
    model_name_or_path="yosefw/colbert-bert-mini",
)

# Step 2: Initialize the Voyager index
index = indexes.Voyager(
    index_folder="pylate-index",
    index_name="index",
    override=True,  # This overwrites the existing index if any
)

# Step 3: Encode the documents
documents_ids = ["1", "2", "3"]
documents = ["document 1 text", "document 2 text", "document 3 text"]

documents_embeddings = model.encode(
    documents,
    batch_size=32,
    is_query=False,  # Ensure that it is set to False to indicate that these are documents, not queries
    show_progress_bar=True,
)

# Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
index.add_documents(
    documents_ids=documents_ids,
    documents_embeddings=documents_embeddings,
)

Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it:

# To load an index, simply instantiate it with the correct folder/name and without overriding it
index = indexes.Voyager(
    index_folder="pylate-index",
    index_name="index",
)

Retrieving top-k documents for queries

Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries. To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores:

# Step 1: Initialize the ColBERT retriever
retriever = retrieve.ColBERT(index=index)

# Step 2: Encode the queries
queries_embeddings = model.encode(
    ["query for document 3", "query for document 1"],
    batch_size=32,
    is_query=True,  #  # Ensure that it is set to False to indicate that these are queries
    show_progress_bar=True,
)

# Step 3: Retrieve top-k documents
scores = retriever.retrieve(
    queries_embeddings=queries_embeddings,
    k=10,  # Retrieve the top 10 matches for each query
)

Reranking

If you only want to use the ColBERT model to perform reranking on top of your first-stage retrieval pipeline without building an index, you can simply use rank function and pass the queries and documents to rerank:

from pylate import rank, models

queries = [
    "query A",
    "query B",
]

documents = [
    ["document A", "document B"],
    ["document 1", "document C", "document B"],
]

documents_ids = [
    [1, 2],
    [1, 3, 2],
]

model = models.ColBERT(
    model_name_or_path="yosefw/colbert-bert-mini",
)

queries_embeddings = model.encode(
    queries,
    is_query=True,
)

documents_embeddings = model.encode(
    documents,
    is_query=False,
)

reranked_documents = rank.rerank(
    documents_ids=documents_ids,
    queries_embeddings=queries_embeddings,
    documents_embeddings=documents_embeddings,
)

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Col BERTTriplet

• Evaluated with <code>pylate.evaluation.colbert_triplet.ColBERTTripletEvaluator</code>

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

msmarco-train

• Dataset: msmarco-train at 6853021
• Size: 983,844 training samples
• Columns: <code>query_id</code>, <code>query</code>, <code>positive</code>, <code>negative_1</code>, <code>negative_2</code>, <code>negative_3</code>, and <code>negative_4</code>
• Approximate statistics based on the first 1000 samples:

  	query_id	query	positive	negative_1	negative_2	negative_3	negative_4
  type	int	string	string	string	string	string	string
  details	<ul><li>24716: ~0.20%</li><li>24720: ~0.20%</li><li>24724: ~0.20%</li><li>24725: ~0.20%</li><li>24729: ~0.20%</li><li>24731: ~0.20%</li><li>24732: ~0.20%</li><li>24734: ~0.20%</li><li>24735: ~0.20%</li><li>24736: ~0.20%</li><li>24737: ~0.20%</li><li>24740: ~0.20%</li><li>24741: ~0.20%</li><li>24743: ~0.20%</li><li>24746: ~0.20%</li><li>24749: ~0.20%</li><li>24751: ~0.20%</li><li>24753: ~0.20%</li><li>24756: ~0.20%</li><li>24757: ~0.20%</li><li>24763: ~0.20%</li><li>24764: ~0.20%</li><li>24765: ~0.20%</li><li>24766: ~0.20%</li><li>24769: ~0.20%</li><li>24772: ~0.20%</li><li>24775: ~0.20%</li><li>24778: ~0.20%</li><li>24782: ~0.20%</li><li>24783: ~0.20%</li><li>24784: ~0.20%</li><li>24785: ~0.20%</li><li>24786: ~0.20%</li><li>24787: ~0.20%</li><li>24789: ~0.20%</li><li>24790: ~0.20%</li><li>24791: ~0.20%</li><li>24792: ~0.20%</li><li>24794: ~0.20%</li><li>24798: ~0.20%</li><li>24799: ~0.20%</li><li>24800: ~0.20%</li><li>24801: ~0.20%</li><li>24812: ~0.20%</li><li>24813: ~0.20%</li><li>24815: ~0.20%</li><li>24816: ~0.20%</li><li>24819: ~0.20%</li><li>24820: ~0.20%</li><li>24823: ~0.20%</li><li>24824: ~0.20%</li><li>24830: ~0.20%</li><li>24833: ~0.20%</li><li>24834: ~0.20%</li><li>24839: ~0.20%</li><li>24840: ~0.20%</li><li>24842: ~0.20%</li><li>24843: ~0.20%</li><li>24844: ~0.20%</li><li>24846: ~0.20%</li><li>24847: ~0.20%</li><li>24849: ~0.20%</li><li>24851: ~0.20%</li><li>24853: ~0.20%</li><li>24859: ~0.20%</li><li>24860: ~0.20%</li><li>24861: ~0.20%</li><li>24864: ~0.20%</li><li>24865: ~0.20%</li><li>24871: ~0.20%</li><li>24874: ~0.20%</li><li>24875: ~0.20%</li><li>24876: ~0.20%</li><li>24880: ~0.20%</li><li>24885: ~0.20%</li><li>24894: ~0.20%</li><li>24896: ~0.20%</li><li>24900: ~0.20%</li><li>24902: ~0.20%</li><li>24910: ~0.20%</li><li>24914: ~0.20%</li><li>24916: ~0.20%</li><li>24921: ~0.20%</li><li>24928: ~0.20%</li><li>24931: ~0.20%</li><li>24934: ~0.20%</li><li>24935: ~0.20%</li><li>24937: ~0.20%</li><li>24939: ~0.20%</li><li>24942: ~0.20%</li><li>24943: ~0.20%</li><li>24944: ~0.20%</li><li>24946: ~0.20%</li><li>24947: ~0.20%</li><li>24949: ~0.20%</li><li>24950: ~0.20%</li><li>24953: ~0.20%</li><li>24955: ~0.20%</li><li>24956: ~0.20%</li><li>24958: ~0.20%</li><li>24959: ~0.20%</li><li>24961: ~0.20%</li><li>24966: ~0.20%</li><li>24967: ~0.20%</li><li>24968: ~0.20%</li><li>24969: ~0.20%</li><li>24970: ~0.20%</li><li>24974: ~0.20%</li><li>24975: ~0.20%</li><li>24976: ~0.20%</li><li>24981: ~0.20%</li><li>24982: ~0.20%</li><li>24983: ~0.20%</li><li>24985: ~0.20%</li><li>24986: ~0.20%</li><li>24987: ~0.20%</li><li>24992: ~0.20%</li><li>24995: ~0.20%</li><li>24997: ~0.20%</li><li>25000: ~0.20%</li><li>25001: ~0.20%</li><li>25014: ~0.20%</li><li>25017: ~0.20%</li><li>25019: ~0.20%</li><li>25021: ~0.20%</li><li>25024: ~0.20%</li><li>25028: ~0.20%</li><li>25029: ~0.20%</li><li>25031: ~0.20%</li><li>25032: ~0.20%</li><li>25033: ~0.20%</li><li>25043: ~0.20%</li><li>25048: ~0.20%</li><li>25050: ~0.20%</li><li>25051: ~0.20%</li><li>25054: ~0.20%</li><li>25060: ~0.20%</li><li>25062: ~0.20%</li><li>25063: ~0.20%</li><li>25064: ~0.20%</li><li>25066: ~0.20%</li><li>25069: ~0.20%</li><li>25070: ~0.20%</li><li>25073: ~0.20%</li><li>25075: ~0.20%</li><li>25078: ~0.20%</li><li>25079: ~0.20%</li><li>25081: ~0.20%</li><li>25084: ~0.20%</li><li>25085: ~0.20%</li><li>25086: ~0.20%</li><li>25089: ~0.20%</li><li>25090: ~0.20%</li><li>25091: ~0.20%</li><li>25092: ~0.20%</li><li>25093: ~0.20%</li><li>25094: ~0.20%</li><li>25096: ~0.20%</li><li>25098: ~0.20%</li><li>25101: ~0.20%</li><li>25102: ~0.20%</li><li>25104: ~0.20%</li><li>25106: ~0.20%</li><li>25109: ~0.20%</li><li>25114: ~0.20%</li><li>25117: ~0.20%</li><li>25118: ~0.20%</li><li>25119: ~0.20%</li><li>25120: ~0.20%</li><li>25122: ~0.20%</li><li>25125: ~0.20%</li><li>25126: ~0.20%</li><li>25130: ~0.20%</li><li>25133: ~0.20%</li><li>25136: ~0.20%</li><li>25137: ~0.20%</li><li>25138: ~0.20%</li><li>25140: ~0.20%</li><li>25143: ~0.20%</li><li>25152: ~0.20%</li><li>25155: ~0.20%</li><li>25157: ~0.20%</li><li>25158: ~0.20%</li><li>25159: ~0.20%</li><li>25160: ~0.20%</li><li>25161: ~0.20%</li><li>25165: ~0.20%</li><li>25168: ~0.20%</li><li>25169: ~0.20%</li><li>25170: ~0.20%</li><li>25172: ~0.20%</li><li>25173: ~0.20%</li><li>25174: ~0.20%</li><li>25176: ~0.20%</li><li>25177: ~0.20%</li><li>25180: ~0.20%</li><li>25187: ~0.20%</li><li>25188: ~0.20%</li><li>25189: ~0.20%</li><li>25190: ~0.20%</li><li>25191: ~0.20%</li><li>25192: ~0.20%</li><li>25193: ~0.20%</li><li>25194: ~0.20%</li><li>25195: ~0.20%</li><li>25197: ~0.20%</li><li>25199: ~0.20%</li><li>25202: ~0.20%</li><li>25204: ~0.20%</li><li>25206: ~0.20%</li><li>25207: ~0.20%</li><li>25208: ~0.20%</li><li>25210: ~0.20%</li><li>25211: ~0.20%</li><li>25213: ~0.20%</li><li>25219: ~0.20%</li><li>25226: ~0.20%</li><li>25229: ~0.20%</li><li>25231: ~0.20%</li><li>25232: ~0.20%</li><li>25233: ~0.20%</li><li>25236: ~0.20%</li><li>25245: ~0.20%</li><li>25246: ~0.20%</li><li>25249: ~0.20%</li><li>25250: ~0.20%</li><li>25253: ~0.20%</li><li>25255: ~0.20%</li><li>25256: ~0.20%</li><li>25257: ~0.20%</li><li>25258: ~0.20%</li><li>25259: ~0.20%</li><li>25262: ~0.20%</li><li>25264: ~0.20%</li><li>25265: ~0.20%</li><li>25268: ~0.20%</li><li>25269: ~0.20%</li><li>25270: ~0.20%</li><li>25271: ~0.20%</li><li>25274: ~0.20%</li><li>25277: ~0.20%</li><li>25278: ~0.20%</li><li>25281: ~0.20%</li><li>25284: ~0.20%</li><li>25285: ~0.20%</li><li>25289: ~0.20%</li><li>25292: ~0.20%</li><li>25293: ~0.20%</li><li>25296: ~0.20%</li><li>25297: ~0.20%</li><li>25298: ~0.20%</li><li>25300: ~0.20%</li><li>25305: ~0.20%</li><li>25308: ~0.20%</li><li>25310: ~0.20%</li><li>25311: ~0.20%</li><li>25312: ~0.20%</li><li>25314: ~0.20%</li><li>25315: ~0.20%</li><li>25316: ~0.20%</li><li>25317: ~0.20%</li><li>25321: ~0.20%</li><li>25323: ~0.20%</li><li>25324: ~0.20%</li><li>25327: ~0.20%</li><li>25330: ~0.20%</li><li>25331: ~0.20%</li><li>25334: ~0.20%</li><li>25336: ~0.20%</li><li>25338: ~0.20%</li><li>25339: ~0.20%</li><li>25342: ~0.20%</li><li>25343: ~0.20%</li><li>25346: ~0.20%</li><li>25348: ~0.20%</li><li>25350: ~0.20%</li><li>25352: ~0.20%</li><li>25355: ~0.20%</li><li>25356: ~0.20%</li><li>25357: ~0.20%</li><li>25358: ~0.20%</li><li>25359: ~0.20%</li><li>25360: ~0.20%</li><li>25361: ~0.20%</li><li>25366: ~0.20%</li><li>25367: ~0.20%</li><li>25370: ~0.20%</li><li>25371: ~0.20%</li><li>25374: ~0.20%</li><li>25379: ~0.20%</li><li>25380: ~0.20%</li><li>25381: ~0.20%</li><li>25383: ~0.20%</li><li>25384: ~0.20%</li><li>25388: ~0.20%</li><li>25389: ~0.20%</li><li>25390: ~0.20%</li><li>25391: ~0.20%</li><li>25392: ~0.20%</li><li>25394: ~0.20%</li><li>25397: ~0.20%</li><li>25401: ~0.20%</li><li>25402: ~0.20%</li><li>25403: ~0.20%</li><li>25405: ~0.20%</li><li>25408: ~0.20%</li><li>25410: ~0.20%</li><li>25411: ~0.20%</li><li>25413: ~0.20%</li><li>25417: ~0.20%</li><li>25420: ~0.20%</li><li>25421: ~0.20%</li><li>25423: ~0.20%</li><li>25428: ~0.20%</li><li>25430: ~0.20%</li><li>25432: ~0.20%</li><li>25433: ~0.20%</li><li>25436: ~0.20%</li><li>25439: ~0.20%</li><li>25445: ~0.20%</li><li>25446: ~0.20%</li><li>25448: ~0.20%</li><li>25450: ~0.20%</li><li>25453: ~0.20%</li><li>25454: ~0.20%</li><li>25456: ~0.20%</li><li>25457: ~0.20%</li><li>25458: ~0.20%</li><li>25466: ~0.20%</li><li>25467: ~0.20%</li><li>25468: ~0.20%</li><li>25469: ~0.20%</li><li>25470: ~0.20%</li><li>25471: ~0.20%</li><li>25473: ~0.20%</li><li>25475: ~0.20%</li><li>25476: ~0.20%</li><li>25477: ~0.20%</li><li>25478: ~0.20%</li><li>25479: ~0.20%</li><li>25480: ~0.20%</li><li>25482: ~0.20%</li><li>25486: ~0.20%</li><li>25487: ~0.20%</li><li>25488: ~0.20%</li><li>25490: ~0.20%</li><li>25494: ~0.20%</li><li>25497: ~0.20%</li><li>25499: ~0.20%</li><li>25500: ~0.20%</li><li>25504: ~0.20%</li><li>25508: ~0.20%</li><li>25511: ~0.20%</li><li>25512: ~0.20%</li><li>25513: ~0.20%</li><li>25514: ~0.20%</li><li>25516: ~0.20%</li><li>25518: ~0.20%</li><li>25519: ~0.20%</li><li>25525: ~0.20%</li><li>25526: ~0.20%</li><li>25528: ~0.20%</li><li>25530: ~0.20%</li><li>25532: ~0.20%</li><li>25533: ~0.20%</li><li>25535: ~0.20%</li><li>25536: ~0.20%</li><li>25539: ~0.20%</li><li>25540: ~0.20%</li><li>25541: ~0.20%</li><li>25544: ~0.20%</li><li>25549: ~0.20%</li><li>25552: ~0.20%</li><li>25553: ~0.20%</li><li>25555: ~0.20%</li><li>25558: ~0.20%</li><li>25560: ~0.20%</li><li>25562: ~0.20%</li><li>25565: ~0.20%</li><li>25566: ~0.20%</li><li>25567: ~0.20%</li><li>25569: ~0.20%</li><li>25571: ~0.20%</li><li>25572: ~0.20%</li><li>25574: ~0.20%</li><li>25575: ~0.20%</li><li>25583: ~0.20%</li><li>25584: ~0.20%</li><li>25587: ~0.20%</li><li>25589: ~0.20%</li><li>25591: ~0.20%</li><li>25593: ~0.20%</li><li>25594: ~0.20%</li><li>25595: ~0.20%</li><li>25598: ~0.20%</li><li>25601: ~0.20%</li><li>25604: ~0.20%</li><li>25605: ~0.20%</li><li>25607: ~0.20%</li><li>25609: ~0.20%</li><li>25612: ~0.20%</li><li>25613: ~0.20%</li><li>25614: ~0.20%</li><li>25615: ~0.20%</li><li>25617: ~0.20%</li><li>25618: ~0.20%</li><li>25619: ~0.20%</li><li>25620: ~0.20%</li><li>25624: ~0.20%</li><li>25628: ~0.20%</li><li>25629: ~0.20%</li><li>25631: ~0.20%</li><li>25633: ~0.20%</li><li>25637: ~0.20%</li><li>25638: ~0.20%</li><li>25639: ~0.20%</li><li>25640: ~0.20%</li><li>25641: ~0.20%</li><li>25642: ~0.20%</li><li>25644: ~0.20%</li><li>25648: ~0.20%</li><li>25649: ~0.20%</li><li>25650: ~0.20%</li><li>25652: ~0.20%</li><li>25653: ~0.20%</li><li>25655: ~0.20%</li><li>25656: ~0.20%</li><li>25657: ~0.20%</li><li>25659: ~0.20%</li><li>25660: ~0.20%</li><li>25661: ~0.20%</li><li>25667: ~0.20%</li><li>25668: ~0.20%</li><li>25669: ~0.20%</li><li>25670: ~0.20%</li><li>25672: ~0.20%</li><li>25673: ~0.20%</li><li>25675: ~0.20%</li><li>25676: ~0.20%</li><li>25678: ~0.20%</li><li>25679: ~0.20%</li><li>25683: ~0.20%</li><li>25684: ~0.20%</li><li>25685: ~0.20%</li><li>25687: ~0.20%</li><li>25691: ~0.20%</li><li>25695: ~0.20%</li><li>25697: ~0.20%</li><li>25699: ~0.20%</li><li>25702: ~0.20%</li><li>25704: ~0.20%</li><li>25705: ~0.20%</li><li>25706: ~0.20%</li><li>25709: ~0.20%</li><li>25710: ~0.20%</li><li>25712: ~0.20%</li><li>25714: ~0.20%</li><li>25715: ~0.20%</li><li>25716: ~0.20%</li><li>25722: ~0.20%</li><li>25725: ~0.20%</li><li>25727: ~0.20%</li><li>25728: ~0.20%</li><li>25730: ~0.20%</li><li>25731: ~0.20%</li><li>25733: ~0.20%</li><li>25735: ~0.20%</li><li>25736: ~0.20%</li><li>25737: ~0.20%</li><li>25738: ~0.20%</li><li>25740: ~0.20%</li><li>25741: ~0.20%</li><li>25742: ~0.20%</li><li>25744: ~0.20%</li><li>25746: ~0.20%</li><li>25748: ~0.20%</li><li>25749: ~0.20%</li><li>25751: ~0.20%</li><li>25752: ~0.20%</li><li>25754: ~0.20%</li><li>25756: ~0.20%</li><li>25760: ~0.20%</li><li>25761: ~0.20%</li><li>25762: ~0.20%</li><li>25766: ~0.20%</li><li>25768: ~0.20%</li><li>25771: ~0.20%</li><li>25772: ~0.20%</li><li>25773: ~0.20%</li><li>25777: ~0.20%</li><li>25778: ~0.20%</li><li>25780: ~0.20%</li><li>25782: ~0.20%</li><li>25783: ~0.20%</li><li>25784: ~0.20%</li><li>25787: ~0.20%</li><li>25788: ~0.20%</li><li>25789: ~0.20%</li><li>25790: ~0.20%</li><li>25791: ~0.20%</li></ul>	<ul><li>min: 6 tokens</li><li>mean: 10.55 tokens</li><li>max: 27 tokens</li></ul>	<ul><li>min: 23 tokens</li><li>mean: 31.97 tokens</li><li>max: 32 tokens</li></ul>	<ul><li>min: 18 tokens</li><li>mean: 31.95 tokens</li><li>max: 32 tokens</li></ul>	<ul><li>min: 19 tokens</li><li>mean: 31.97 tokens</li><li>max: 32 tokens</li></ul>	<ul><li>min: 20 tokens</li><li>mean: 31.97 tokens</li><li>max: 32 tokens</li></ul>	<ul><li>min: 21 tokens</li><li>mean: 31.97 tokens</li><li>max: 32 tokens</li></ul>

• Samples:

  query_id	query	positive	negative_1	negative_2	negative_3	negative_4
  <code>24716</code>	<code>are rock band guitars compatible with guitar hero</code>	<code>Instruments compatible with The Beatles: Rock Band, Green Day: Rock Band, Lego Rock Band, Guitar Hero 4, Guitar Hero 5, Guitar Hero: Van Halen, Guitar Hero Smash Hits, Guitar Hero: Metallica, and Band Hero.</code>	<code>However, guitar compatibility has become a major question now that Rock Band has joined Guitar Hero in the genre. You can't mix and match controllers with games anymore now that we have two competing game franchises with different guitar designs.</code>	<code>Games compatible with the Guitar Hero 4, Band Hero, Rock Band 1, Rock Band 2, and The Beatles: Rock Band mics.</code>	<code>Our position is really all about respect for our consumers and for the money that they have spent to get into the game space in the first place. They spend a fortune on games: on consoles, on hardware, and we're sensitive to that, he said.</code>	<code>For more on the music party game resurgence, check out IGN's discussion about both Guitar Hero and Rock Band. Cassidee is a freelance writer for various outlets around the web. You can chat with her about all things geeky on Twitter.</code>
  <code>24716</code>	<code>are rock band guitars compatible with guitar hero</code>	<code>Instruments compatible with The Beatles: Rock Band, Green Day: Rock Band, Lego Rock Band, Guitar Hero 4, Guitar Hero 5, Guitar Hero: Van Halen, Guitar Hero Smash Hits, Guitar Hero: Metallica, and Band Hero.</code>	<code>While the Xbox 360 RedOctane controllers play nicely with Rock Band, we can't say that the Rock Band guitar returns the favor. The Rock Band Fender Stratocaster will not work with Guitar Hero II or Guitar Hero III.</code>	<code>The new peripherals for the recently-announced Guitar Hero Live will not work with Rock Band 4, although Sussman hasn't completely ruled it out, saying he won't know for sure until [they] get one in the office..</code>	<code>Note: Rock Band instruments ARE compatible to Guitar Hero 5 and Band Hero for the Wii Edit. Do NOT change those things. Rock Band series Guitars, Drumkits, and Microphones ARE, and I mean ARE compatible fore Guitar Hero 5 and Band Hero for the Wii. THEY FUCKING ARE!</code>	<code>Speaking with Business Insider, Rock Band 4 project director Daniel Sussman revealed the upcoming music game will be compatible with last-gen Guitar Hero and Rock Band peripherals. Sussman explains he and his team wanted to see this integration through as a way to cut costs for consumers.</code>
  <code>24720</code>	<code>are rocks found in temperate deciduous forests</code>	<code>Since the smaller biomes make up the Deciduous Forests it shows that because it is a forest then naturally trees, animals and rocks are found inside of it. (Riley M) Temperate forests are in cool rainy areas that have trees that lose their leaves in the Fall, and then grow back in the Spring.</code>	<code>Plants Animals Climate Northeast Asian Deciduous Forest Deciduous forests can be found in the eastern half of North America, and the middle of Europe. There are many deciduous forests in Asia. Some of the major areas that they are in are southwest Russia, Japan, and eastern China.</code>	<code>Northeast Asian Deciduous Forest. Deciduous forests can be found in the eastern half of North America, and the middle of Europe. There are many deciduous forests in Asia. Some of the major areas that they are in are southwest Russia, Japan, and eastern China.</code>	<code>The trees found in temperate forest are called hardwoods. Hardwoods are trees that loose their leaves in the winter, also their trunks are also made of a bark that is very hard. (Dimitri T.). Most of the trees in temperate forest are maple, birch, beech, oak, hickory, and sweet gum.</code>	<code>Temperate (Deciduous Forest) occupies the eastern part of the United States {Coraima T}. Temperate deciduous forests are found in continents such as North America, South America, Europe, Asia, Austrailia, and Africa. The soil in these temperate forests can be very rocky and sandy.</code>

• Loss: <code>pylate.losses.contrastive.Contrastive</code>

Evaluation Dataset

msmarco-train

• Dataset: msmarco-train at 6853021
• Size: 20,000 evaluation samples
• Columns: <code>query_id</code>, <code>query</code>, <code>positive</code>, <code>negative_1</code>, <code>negative_2</code>, <code>negative_3</code>, and <code>negative_4</code>
• Approximate statistics based on the first 1000 samples:

  	query_id	query	positive	negative_1	negative_2	negative_3	negative_4
  type	int	string	string	string	string	string	string

The model card is truncated. Read the rest on Hugging Face.