reign-base-l3_gn-gte-large_dapfam-ftwarm-c512s512

reign-base-l3_gn-gte-large_dapfam-ftwarm-c512s512

REIGN base-l3 cross-chunk encoder over a frozen GTE-large guidance network, fine-tuned on the DAPFAM patent retrieval task (warm-start fine-tune, chunk 512 / stride 512).

Configuration

Reported results

nDCG@100 on DAPFAM, top-k = 100 over the full FullText corpus with self-matches removed. These are the values the paper reports for this exact checkpoint (Appendix J).

Fine-tuning does not exceed zero-shot on this task. The paper's finding is that no cell in the sweep improves on the matched zero-shot backbone, and that naive fine-tuning at lr 1e-5 degrades it by 0.4–1.5 points. This family is released so the negative result is inspectable, not as a recommended starting point — for patent retrieval, prefer the zero-shot GoodWiki-Long checkpoints.

Usage

The checkpoint holds only the REIGN cross-chunk encoder. The guidance network is loaded separately and stays frozen, so both must be named at construction time.

pip install git+https://github.com/devrimcavusoglu/reign.git

import numpy as np
from huggingface_hub import snapshot_download
from reign.encoders.reign import ReignBaselineEncoder

checkpoint_path = snapshot_download("devrim/reign-base-l3_gn-gte-large_dapfam-ftwarm-c512s512")

encoder = ReignBaselineEncoder(
    checkpoint_path=checkpoint_path,
    gn_model="thenlper/gte-large",
    chunk_size=512,
    stride=512,
)
docs = [open("doc_a.txt").read(), open("doc_b.txt").read()]
emb = encoder.encode(docs, batch_size=8)   # (2, hidden_size), L2-normalised
print(float(np.dot(emb[0], emb[1])))       # cosine similarity

ReignBaselineEncoder returns L2-normalised vectors, so the cosine is a dot product. chunk_size is the guidance network's sliding-window size — 512 for every released checkpoint, matching its context window — and stride controls the overlap, with stride == chunk_size giving non-overlapping chunking. The evaluation-time stride is a runtime argument, and the paper's headline tables report the best-performing stride per guidance network.

For the lower-level surface, ReignModel (a PreTrainedModel consuming inputs_embeds) and ReignFeatureExtractor (the guidance-network wrapper, with the on-disk embedding cache) are importable from reign and reign.feature_extractor.

Operating regime

REIGN targets multi-chunk inputs and primarily document-to-document retrieval. Inputs shorter than the chunk size collapse to a single chunk embedding, leaving the cross-chunk encoder nothing to aggregate — that regime is served by the guidance network alone, and this checkpoint should not be used for it.

Training recipe

DAPFAM's relevance labels are binary, so this family uses a standard query/positive/negative contrastive path rather than the graded three-way cosine objective of the GoodWiki-Long checkpoints.

The full sweep covers lr ∈ {1e-5, 5e-6, 2e-6, 1e-6} and weight decay ∈ {1e-4, 1e-2, 1e-1}. See docs/TRAINING.md in the code repository.

Because 16-mixed training is not bit-reproducible even at a fixed seed, a retrained checkpoint will not match these weights bit-for-bit; compare metrics, not weights.

Files

• config.json — ReignModel configuration
• model.safetensors — encoder weights (float32)

Links

• Code: https://github.com/devrimcavusoglu/reign
• Project page: https://devrimcavusoglu.github.io/reign
• Dataset: https://huggingface.co/datasets/devrim/goodwiki_long_synthetic_ir
• Paper: REIGN: Refurbished Embeddings with Integrated Guidance Networks for Efficient Context-Length Scaling, Findings of the Association for Computational Linguistics: EMNLP 2026 (to appear).

Citation

@inproceedings{cavusoglu2026reign,
  title     = {{REIGN}: Refurbished Embeddings with Integrated Guidance Networks for Efficient Context-Length Scaling},
  author    = {{\c{C}}avu{\c{s}}o{\u{g}}lu, Devrim and Akba{\c{s}}, Emre},
  booktitle = {Findings of the Association for Computational Linguistics: {EMNLP} 2026},
  year      = {2026},
  publisher = {Association for Computational Linguistics},
  note      = {To appear}
}

License

Apache License 2.0. The devrim/goodwiki_long_synthetic_ir dataset is released under CC BY-SA 4.0, preserving the share-alike licensing and attribution of GoodWiki and the underlying Wikipedia text.