Downloads · 30 days
42
2% of all-time downloads
nasa-impact/nasa-smd-ibm-ranker
nasa-smd-ibm-ranker is a text classification model from nasa-impact. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.
nasa-smd-ibm-ranker (INDUS Ranker) is a encoder based model that takes a search query, and a passage, and calculates the relevancy of the passage to the query. This is used in conjunction with sentence transformers to…
Downloads · 30 days
42
2% of all-time downloads
All-time downloads
1.7K
Public
Repo size
997 MB
Likes
3
Public
Click a slice to open those files.
.bin499 MB · 99%
From the Hugging Face model README
nasa-smd-ibm-ranker (INDUS Ranker) is a encoder based model that takes a search query, and a passage, and calculates the relevancy of the passage to the query. This is used in conjunction with sentence transformers to re-rank the passages matched by the sentence transformer, there-by improving relevance of Information Retrieval processes.
The Model is Finetuned using MS-Marco, and tested using Science QA datasets.
The Model is an integral part of Neural Search Information Retreival process used by the Science Discovery Engine, Along with the finetuned sentence transformer (https://huggingface.co/nasa-impact/nasa-smd-ibm-st-v2).
Model Evaluation on msmarco dev set, and NASA-QA:

Both query and passage have to fit in 512 Tokens (along with [CLS] and [SEP] special tokens). The intended use is to rerank the first dozens of embedding search results.
from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained("nasa-impact/nasa-smd-ibm-ranker")
model = AutoModelForSequenceClassification.from_pretrained("nasa-impact/nasa-smd-ibm-ranker")
This Ranker Model is released in support of the training and evaluation of the encoder language model "Indus".
Accompanying paper can be found here: https://arxiv.org/abs/2405.10725
@misc {nasa-impact_2024,
author = { {NASA-IMPACT} },
title = { nasa-smd-ibm-ranker (Revision 4f42d19) },
year = 2024,
url = { https://huggingface.co/nasa-impact/nasa-smd-ibm-ranker },
doi = { 10.57967/hf/1849 },
publisher = { Hugging Face }
}