First_Model

Model Card for FIRST

<!-- Provide a quick summary of what the model is/does. -->

FIRST is a language models trained for listwise reranking tasks leveraging the output logits of the first generated identifier to directly produce a ranked ordering of candidates. Built on the Zephyr-7B-β model, FIRST undergoes single-stage fine-tuning on a converted alphabetic version of the RankZephyr dataset. (i,e RankGPT-4 reorderings of OpenAI's Ada2 orderings for 5k queries) More details can be found in the paper.

Model Description

<!-- Provide a longer summary of what this model is. -->

• Model type: Fine-tuned on listwise reranking data from Zephyr-7B-β model.
• Language(s) (NLP): English
• License: MIT
• Finetuned from model: HuggingFaceH4/zephyr-7b-beta

Model Sources

<!-- Provide the basic links for the model. -->

• Repository: https://github.com/gangiswag/llm-reranker
• Paper: https://arxiv.org/abs/2406.15657

Evaluations

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

At the time of release, FIRST demonstrates superior performance across a variety of reranking datasets. The table below provides a detailed performance comparison against other LLM rerankers on the BEIR benchmark. (More details can be found in the paper.)

Citation

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

If you find FIRST useful for your work, please consider citing our paper:

@article{reddy2024first,
  title={FIRST: Faster Improved Listwise Reranking with Single Token Decoding},
  author={Reddy, Revanth Gangi and Doo, JaeHyeok and Xu, Yifei and Sultan, Md Arafat and Swain, Deevya and Sil, Avirup and Ji, Heng},
  journal={arXiv preprint arXiv:2406.15657},
  year={2024}
}