search-reranker-irishgov-l6-fast-v1

search-reranker-irishgov-l6-fast-v1

Fast 6-layer bilingual reranker for Irish government search.

This model is the current fast-path reranker line from the openMed reranker cycle. It is derived from cross-encoder/mmarco-mMiniLMv2-L12-H384-v1, reduced to 6 layers, then fine-tuned on a bilingual Irish government-oriented mix.

Intended use

• Best use: stage-1 prefilter before a stronger reranker, or direct reranking when the effective candidate set is about 10-20.
• Strong current serving pattern: l6-fast -> top10 -> search-reranker-broad-v1.
• Not a full replacement for the broad reranker on large 80-candidate pools when maximum final quality matters.

Why publish this line

The broad reranker remains stronger on large candidate pools, but it is too expensive in the critical path when reranking large candidate sets on CPU.

This 6-layer line is valuable because:

• it is much faster on CPU q8
• it remains strong on the bilingual proxy and office-holder slices
• it is the current best stage-1 model for the winning cascade

Key q8 benchmarks

Best current cascade using this model

On the 80-candidate in-domain holdout, the strongest measured serving path in this cycle is:

• l6-fast q8 -> top10 -> search-reranker-broad-v1 q8

Measured result:

• overall MRR@10 = 0.9433
• Irish MRR@10 = 0.8967
• English MRR@10 = 0.9900
• cascade query throughput = 15.20 qps

That cascade currently beats full broad reranking on the same 80-candidate holdout while being much faster.

Artifacts

• Raw HF checkpoint: ./
• ONNX dynamic q8 artifact: onnx/model_quantized.onnx

Training provenance

• Base model: cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
• Training mix: bilingual Irish/English proxy mix plus Irish government office-holder supervision
• Training metadata: training_info.json

Caveats

• The q8 artifact is optimized for CPU inference, not for maximum compression ratio.
• This model is best treated as a fast prefilter or low-latency reranker, not the final highest-quality reranker on very large candidate sets.

Portfolio comparison

<!-- portfolio-comparison:start -->

Updated 2026-03-20 from local reranker reports only.

Use this section as the side-by-side public temsa reranker view. Cells are intentionally left out or summarized when the local report set does not contain a trustworthy non-quantized benchmark for the same public path.

General bilingual rerankers

Policy rerankers

Dynamic q8 CPU at K=20

These rows use the current local K=20 benchmark harness:

• proxy_k20: policy=none
• broad_policy_k20: policy=gov_broad_v1
• office_holder_k20: policy=office_holder_v1
• fixed threads=10, batch_size=32

Intentional gaps:

• search-reranker-broad-v1: the local reports include strong fp32 ONNX proxy and office data, but not a matching fp32 in-domain finephrase / hard-K10 / holdout-A03 set, so those are not claimed here.
• search-reranker-irishgov-l5-k10-v2: the local non-quantized reports are trustworthy for office / finephrase / hard-K10, but not for the same proxy runtime shape as the shipped q8 path.
• search-reranker-irishgov-l6-fast-v1: the local non-quantized reports cover proxy and office, but not the fresh holdout-A03 slice used for the q8 K=10 comparison.

<!-- portfolio-comparison:end -->