synthsearch-qwen3-8b-sft-v1

SynthSearch-Qwen3-8B: historical LoRA adapter for the CER-Bench search controller

Status: historical artifact, not re-evaluated. This adapter was trained in April 2026 on the historical CER-Bench task collection. The companion paper, Fixed Rankings, Moving Leaders: A Relevance-Label and Source-Identity Audit of Scientific Retrieval Evaluation (Bansal & Gangwani, 2026; code and paper at https://github.com/aayambansal/CER-Bench), later showed that (i) the corpus those tasks were built on carried a source-identity defect (cited-paper identifiers substituted for articles' own identifiers; 247 PMC identifiers changed on re-fetch) and (ii) the collection's system ordering depends on which relevance labels are used (9 of 45 pairwise orderings reverse between seed and pooled labels with all rankings held fixed). Any score historically reported for this adapter inherits both caveats and should not be read as a validated biomedical retrieval result. The adapter is released for transparency and reproducibility of the historical record.

What it is

A PEFT/LoRA adapter on Qwen/Qwen3-8B fine-tuned by supervised learning to act as an iterative literature-search controller: given a biomedical question, emit search queries, read results, refine, and stop with an evidence set. The controller interface (SEARCH / REFINE / STOP actions over a BM25 tool) is the one recorded in the historical agent traces at https://huggingface.co/datasets/aayambansall/CER-Bench (traces/agent/).

Training data

data/training/training_tasks.jsonl in the GitHub repository: 1,000 corpus-grounded synthetic tasks generated by a proprietary model (claude-sonnet-4-20250514) over the historical 4,936-record PubMed corpus, with seed supporting-document identifiers as targets (Apache-2.0). Those identifiers are the historical, identity-conflicted ones; they were not migrated to the corrected corpus.

Training configuration (from adapter_config.json / trainer_state.json)

• LoRA rank 32, alpha 64, dropout 0.05; target modules q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• 3 epochs, 306 optimizer steps, per-device train batch size 2, peak learning rate 1.125e-4 (logged at step 10)
• Repository also contains optimizer/scheduler/RNG state from the final checkpoint

Intended use

Research inspection of a historical controller and reproduction of the historical record only. Not for clinical or biomedical decision support. No claim of retrieval accuracy on any corpus is made here.

Citation

@article{bansal2026fixedrankings,
  title  = {Fixed Rankings, Moving Leaders: A Relevance-Label and Source-Identity Audit of Scientific Retrieval Evaluation},
  author = {Bansal, Aayam and Gangwani, Ishaan},
  year   = {2026},
  note   = {Preprint. https://github.com/aayambansal/CER-Bench}
}