search-reranker-irishgov-l6-fast-v5

search-reranker-irishgov-l6-fast-v5

Fast K=10 reranker serving release for Irish gov search.

This is a serving-profile release over temsa/search-reranker-irishgov-l6-fast-v4:

• same raw weights
• same q8 per-channel ONNX artifact family
• faster broad-policy route at max_length=104
• office-holder route stays at max_length=128

Recommended serving profiles

Broad/current/news queries:

• policy: gov_broad_v1
• backend: onnx
• threads: 10
• max length: 104

Office-holder identity queries:

• policy: office_holder_v1
• backend: onnx
• threads: 10
• max length: 128

Key q8 results

Representative throughput:

• corrected holdout v5 (104): 55.77 qps
• fresh holdout v6 (104): 33.34 qps
• office valid (128): 70.91 qps
• office holdout v3 (128): 66.10 qps

Portfolio comparison

<!-- portfolio-comparison:start -->

Updated 2026-03-20 from local reranker reports only.

Use this section as the side-by-side public temsa reranker view. Cells are intentionally left out or summarized when the local report set does not contain a trustworthy non-quantized benchmark for the same public path.

General bilingual rerankers

Policy rerankers

Dynamic q8 CPU at K=20

These rows use the current local K=20 benchmark harness:

• proxy_k20: policy=none
• broad_policy_k20: policy=gov_broad_v1
• office_holder_k20: policy=office_holder_v1
• fixed threads=10, batch_size=32

Intentional gaps:

• search-reranker-broad-v1: the local reports include strong fp32 ONNX proxy and office data, but not a matching fp32 in-domain finephrase / hard-K10 / holdout-A03 set, so those are not claimed here.
• search-reranker-irishgov-l5-k10-v2: the local non-quantized reports are trustworthy for office / finephrase / hard-K10, but not for the same proxy runtime shape as the shipped q8 path.
• search-reranker-irishgov-l6-fast-v1: the local non-quantized reports cover proxy and office, but not the fresh holdout-A03 slice used for the q8 K=10 comparison.

<!-- portfolio-comparison:end -->