ZaraRerank

ZaraRerank

A cross-encoder reranker for the Standard Works of The Church of Jesus Christ of Latter-day Saints. Paired with ZaraEmbed, it puts the right verse first for 243 of 289 held-out Restoration-scripture questions, where OpenAI's embedder manages 126. ZaraRerank reads the question and a candidate verse together and scores how well the verse answers it. It is ZaraBERTa fine-tuned as a reranker for ZaraEmbed's top 100 candidates.

Trained by Brigham Young University-Idaho on public-domain text. Chapter headings, footnotes and Church study aids were not used.

ZaraAI family

Part of ZaraAI, BYU-Idaho's scripture-native model collection:

Purpose

These models are released for research on scripture and on domain-specific language models. They are also released as aids for personal scripture study. They are offered in the hope that they help readers find and understand the Book of Mormon, and come unto Christ.

What is new

1. The right verse first, far more often

The held-out tests use 289 verses from 37 Book of Mormon, D&C and Pearl of Great Price chapters that no training pair touched. A separate language model, not one used for the training questions, wrote one question per verse in two styles, and the same 289 verses appear in both tests:

• no-names: a teenager's search that avoids the verse's names and distinctive words.
• story: a reader naming the speaker, audience or event.

Right verse first (top-1). The comparison is the full ZaraEmbed + ZaraRerank pipeline against OpenAI text-embedding-3-large alone, on bare verse text:

On no-names, the pipeline's lead over OpenAI is significant at McNemar p = 2e-26.

Reranking lifts ZaraEmbed's own top-1 by 52 questions on no-names and 47 on story.

2. On the human-written benchmark

The 152-question benchmark holds human-written study questions across all five volumes. On its Book of Mormon, D&C and Pearl of Great Price questions, the pipeline moves ahead of OpenAI; across all five volumes, OpenAI still leads:

On the 63 Restoration questions, reranking lifts ZaraEmbed from 36 to 47; the lead over OpenAI with chapter context (47 vs 41) is not significant (p = 0.21).

3. Fast enough for search

Scoring 100 candidates takes about 0.23 s per question on an NVIDIA GB10 GPU.

Usage

AutoModelForSequenceClassification with a single logit; a higher logit means a more relevant verse. Prefix both the question and the verse with a single space, and truncate only the verse, at 128 tokens total.

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tok = AutoTokenizer.from_pretrained("BYU-Idaho/ZaraRerank")
model = AutoModelForSequenceClassification.from_pretrained("BYU-Idaho/ZaraRerank").eval()

question = "Why does helping other people count as serving God?"
candidates = [
    "And behold, I tell you these things that ye may learn wisdom; that ye may learn that when ye are in the service of your fellow beings ye are only in the service of your God.",  # Mosiah 2:17
    "Behold, ye have called me your king; and if I, whom ye call your king, do labor to serve you, then ought not ye to labor to serve one another?",  # Mosiah 2:18
]
batch = tok([" " + question] * len(candidates), [" " + v for v in candidates],
            truncation="only_second", max_length=128, padding=True, return_tensors="pt")
with torch.no_grad():
    scores = model(**batch).logits.squeeze(-1)
print(scores.tolist())
# [10.163, -1.638]  -> Mosiah 2:17 ranks above its same-chapter neighbour 2:18 (observed on CPU)

In a search pipeline, retrieve the top 100 verses with ZaraEmbed, score each with ZaraRerank, and sort by logit.

Model details

• Architecture: RobertaForSequenceClassification, 24 layers, hidden 1024, cased, vocabulary 52,176 (ZaraBERTa's), one output logit.
• Base: BYU-Idaho/ZaraBERTa.
• Training data: 68,972 groups of 1 gold verse + 5 negatives drawn from ZaraEmbed's top 50, with NV-Retriever positive-aware filtering (negatives scoring above 95% of the gold are dropped) and up to 2 same-chapter negatives for story-style questions.
• Objective: listwise softmax cross-entropy over each group.
• Training: 1 epoch, batch 16 groups, lr 2e-5, max 128 tokens, about 1 hour on one NVIDIA GB10. Single training seed.
• reranker.json records the training settings.

Intended uses

• Reranking ZaraEmbed's top 100 verses for a question.
• The retrieval stage of a scripture study assistant, where the verse placed first matters most.

Limitations and responsible use

• These models are study aids, not doctrinal authorities, and they are not an official publication of The Church of Jesus Christ of Latter-day Saints.
• Retrieval scores measure textual relevance, not interpretation; read every verse in its context.
• Results come from a single training seed, and the held-out questions were written by a language model.

Provenance and license

Base model roberta-large by Meta AI, MIT license, through ZaraBERTa; that notice applies to any redistribution of these weights. Training text: public-domain editions of the Old Testament, New Testament, Book of Mormon, Doctrine and Covenants and Pearl of Great Price. Training questions were generated by language models over that text. Chapter headings, footnotes and Church study aids were not used. Trained and evaluated by Brigham Young University-Idaho, 2026. Released as open weights under the MIT license.

All pipeline numbers are for ZaraEmbed + ZaraRerank compared with OpenAI's embedder alone.

Citation

@misc{zararerank2026,
  title  = {ZaraRerank: a scripture-native cross-encoder reranker for the Standard Works},
  author = {Vallejo, Ron and Brigham Young University-Idaho},
  year   = {2026},
  url    = {https://huggingface.co/BYU-Idaho/ZaraRerank}
}