Downloads · 30 days
41
32% of all-time downloads
distilled-voyage-modernbert
amal1994/distilled-voyage-modernbert
distilled-voyage-modernbert is a machine learning model from amal1994. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
A ModernBERT-based cross-encoder fine-tuned for re-ranking UK legislation passages. This model distills knowledge from the proprietary Voyage rerank-2.5 API, providing a fully open-source alternative for statutory inf…
All-time downloads
127
Public
Parameters
150M
598 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors598 MB · 99%
At a glance
- Model type
- modernbert
- Access
- Public
- Created
- Nov 14, 2025
- Updated
- Aug 31, 2026
- SHA
- 69de9678
Model card
From the Hugging Face model README
Distilled Voyage ModernBERT Cross-Encoder for UK Statutory Retrieval
A ModernBERT-based cross-encoder fine-tuned for re-ranking UK legislation passages. This model distills knowledge from the proprietary Voyage rerank-2.5 API, providing a fully open-source alternative for statutory information retrieval.
Model Details
- Base Model:
nomic-ai/modernbert-embed-base - Teacher Model: Voyage
rerank-2.5 - Training Data: 5,221 query-passage-score triples from UK legislation (2020-2024)
- Task: Regression-based relevance scoring for legal passage ranking
Use in a Two-Stage Pipeline
- Retrieve the top-100 candidates
- Re-rank them with this cross-encoder
- Return the top-k highest-scoring passages
Training
- Architecture: ModernBERT cross-encoder (regression head)
- Loss: Mean squared error (MSE)
- Training: 3 epochs, batch size 4, learning rate 2e-5
- Max Length: 2,048 tokens
Resources
- Corpus, benchmark and distillation set: https://doi.org/10.15128/r14x51hj064
- Code: https://github.com/alshehriamal1994/uk-statute-retrieval
- Type
- modernbert
- Created
- Nov 14, 2025
- Updated
- Aug 31, 2026