search-query-net

search-query-net

A small neural network that reformulates a natural-language question into a keyword search query that retrieves the passage containing the answer. Trained end-to-end with REINFORCE against a BM25 retrieval reward — the reward is simply "did the generated query pull back the gold passage?", computed exactly from SQuAD labels (no human labels, no LLM judge).

This repo is a rigorous, from-scratch re-implementation of RL query reformulation in the spirit of Nogueira & Cho, Task-Oriented Query Reformulation with Reinforcement Learning (2017). It is a research / learning artifact, not a production web-search system.

What's here

Results (honest protocol)

Held-out test split (title-disjoint from train/val), one shared 20,958-passage BM25 index, selected on val / reported once on test, 3 seeds. Metric = recall@5 (gold passage in top-5).

The autoregressive decoder with an IDF/question-order warm-start is the big lever: it produces a strong single query and closes almost the entire gap to the BM25 oracle on this corpus.

Honest caveats (see DECISIONS.md for the full story)

• Corpus is small (~21k passages) and the answer passage is guaranteed present — real web search is 10⁹–10¹² docs with dense retrieval + rerankers. These numbers do not transfer to open-web search.
• The learned selector does not yet beat "always use head 0": the per-head teachers made heads diverse but imbalanced. Making heads equally-strong-but-different is the clear next step to cash in the best-of-heads headroom.
• A soft head-diversity penalty was tried and did not diversify heads (logged as a negative result).

Reproduce

pip install torch transformers datasets scipy numpy
python prepare_data.py                       # build leak-free SQuAD data + BM25 corpus
python check_data.py                         # assert no train/val/test leakage
# best config (AR pointer-decoder + warm-start), confirmed recall@5 = 0.834 ± 0.002
python runner.py '{"arch":"ar_pointer","allow_expansion":false,"warm_start_steps":200,"teacher":"idf","baseline":"rloo","reward_mode":"gold_rank","temp_consistent":true,"steps":600,"save_ckpt":true}'

Training is fast (~a few minutes on one GPU). allow_expansion:true enables the generation branch so the model can emit tokens not in the question (query expansion) — the path to exceed the copy-only BM25 oracle; still experimental.

Method in one paragraph

The encoder reads the question; the decoder emits a short query. Reward = BM25 retrieval outcome (gold-passage rank), optimized with REINFORCE using a per-question leave-one-out (RLOO) baseline. A pointer-generator mixes a copy distribution over the question's own tokens with a generation distribution over the full vocabulary, gated by a learned scalar — copy is the safe default, generation enables expansion. A supervised warm-start (imitate a BM25-good query built from the question's informative terms) solves the sparse-reward cold-start before RL takes over.