Downloads · 30 days
17
3% of all-time downloads
RankMistral100
liuwenhan/RankMistral100
RankMistral100 is a text ranking model from liuwenhan. Use it for the text ranking task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
We release the full ranking model RankMistral100 distilled from GPT-4o-2024-08-06 used in Sliding Windows Are Not the End: Exploring Full Ranking with Long-Context Large Language Models.
All-time downloads
636
Public
Parameters
7.2B
14.5 GB on disk
Likes
3
Public
Repo makeup
Click a slice to open those files.
.safetensors14.5 GB · 100%
At a glance
- Task
- Text Ranking
- License
- mit
- Model type
- mistral
- Access
- Public
- Created
- Dec 18, 2024
- Updated
- Apr 12, 2025
- SHA
- 1140285f
Base models
Model card
From the Hugging Face model README
Model Information
We release the full ranking model RankMistral100 distilled from GPT-4o-2024-08-06 used in Sliding Windows Are Not the End: Exploring Full Ranking with Long-Context Large Language Models.
<p align="left"> Useful links: 📝 <a href="https://arxiv.org/abs/2412.14574" target="_blank">Paper</a> • 🤗 <a href="https://huggingface.co/datasets/liuwenhan/msmarco_full_ranking_list" target="_blank">Dataset</a> • </a> 🧩 <a href="https://github.com/8421BCD/fullrank" target="_blank">Github</a> </p>Training framework
Our full ranking model aims to directly rerank 100 passages at a time, abandoning the sliding window strategy. We propose a multi-pass sliding window approach for generating the full ranking list as label and design a importance-aware training loss for optimization. <img src="https://8421bcd.oss-cn-beijing.aliyuncs.com/img/image-20241218200920116.png" alt="image-20241218200920116" style="zoom: 45%;" />
Backbone Model
RankMistral100 is finetuned from https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3.
Performance
We surpuss the strong baseline RankZephyr with 1.2 points on BEIR Avg.
| Models | Covid | DBPedia | SciFact | NFCorpus | Signal | Robust04 | Touche | News | Avg. |
|---|---|---|---|---|---|---|---|---|---|
| BM25 | 59.47 | 31.80 | 67.89 | 33.75 | 33.04 | 40.70 | 44.22 | 39.52 | 43.80 |
| monoBERT (340M) | 73.45 | 41.69 | 62.22 | 34.92 | 30.63 | 44.21 | 30.26 | 47.03 | 45.55 |
| monoT5 (220M) | 75.94 | 42.43 | 65.07 | 35.42 | 31.20 | 44.15 | 30.35 | 46.98 | 46.44 |
| RankVicuna (7B) | 79.19 | 44.51 | 70.67 | 34.51 | 34.24 | 48.33 | 33.00 | 47.15 | 48.95 |
| RankZepeyer (7B) | 82.92 | 44.42 | 75.42 | 38.26 | 31.41 | 53.73 | 30.22 | 52.80 | 51.15 |
| RankMistral<sub>100</sub> (7B) | 82.24 | 43.54 | 77.04 | 39.14 | 33.99 | 57.91 | 34.63 | 50.59 | 52.40 |
🌹 If you use this model, please ✨star our <a href="https://github.com/8421BCD/fullrank" target="_blank">GitHub repository</a> to support us. Your star means a lot!
- Task
- Text Ranking
- Type
- mistral
- License
- mit
- Languages
- en
- Created
- Dec 18, 2024
- Updated
- Apr 12, 2025