torah-embed

torah-embed

A 768-dimensional embedding model for retrieval over the Babylonian Talmud, fine-tuned from bge-base-en-v1.5 on a relevance dataset compiled in the sixteenth century.

Ships as a complete RAG: model, corpus, prebuilt index, query script, and a persona file (RABBI.md) that assembles retrieved passages into an answer. Point your Claude at RABBI.md and ask a question — see below.

The idea

Printed in the margin of every page of the Vilna Talmud is a cross-reference apparatus, ein mishpat / ner mitsvah, mapping each passage of talmudic argument to the places in the medieval legal codes that rule from it.

In modern terms it is a hand-curated, cross-register relevance dataset:

• curated by domain experts over centuries, with no model in the loop;
• cross-register by construction — terse codified law on one side, discursive dialectic on the other, sharing little surface vocabulary;
• segment-level, mean 2.6 relevant targets per query;
• and it encodes exactly the deployment task: where does this ruling come from?

50,214 such pairs were extracted from the Sefaria link graph; 27,013 connect Maimonides' Mishneh Torah to the Talmud with English on both sides. This model is trained on them.

Results

552 queries from two entirely held-out tractates (Sanhedrin, Yevamot), retrieved over all 81,481 English Bavli segments — no pruning, no tractate hint. MRR:

R@10 on conceptual questions: 0.161 → 0.221 (+37.1%).

The gradient is the evidence

Improvement rises as lexical overlap falls — the model gained most exactly where surface matching helps least. A memorising model shows the opposite gradient.

Pre-registered criterion

Fixed before training: beat zero-shot on held-out question queries by a margin exceeding the ruling-query gain. A model improving only on rulings has learned the benchmark, not the task.

Use it as a RAG: point your Claude at RABBI.md

This repo ships a complete retrieval system, not just weights: the model, the corpus, a prebuilt index over all 81,481 segments, a query script, and RABBI.md — a persona file that turns raw search results into an answer.

# 1. get everything
pip install sentence-transformers numpy
hf download RobBobin/torah-embed --local-dir torah-embed

# 2. ask the index directly
python3 torah-embed/scripts/ask.py "Can I pray in the bathroom?" -k 8

[0.753] Berakhot 26a:3
  ...Rav Ḥisda said: One may stand opposite a bathroom and pray...

[0.703] Shabbat 10b:1
  the bathroom is different, as it is disgusting. Once a place is called a
  bathroom it is disgusting and no longer fit for prayer...

That is retrieval. It gives you passages, not an answer.

The RAG part

Open Claude Code (or any agent that can read files and run commands) in the directory, and say:

Read RABBI.md and follow it. Then answer: <your question>

The agent will run several searches from different angles, read what comes back, discard the misses, and assemble the passages into coherent paragraphs — quoting the text, preserving disagreements between the Sages rather than averaging them away, and clearly separating what the Gemara says from what it makes of it.

albert.txt is a full worked example: one question, six searches, the answer that resulted, and an index of every passage retrieved.

What RABBI.md is

A teacher in the manner of the Accidental Talmudist — not a rabbinic authority, but someone who reads the page and tells you what is in it and what they make of it. It is permitted to interpret, and required to show the seam:

Berakhot 26a says one may stand opposite a bathroom and pray. ← the text

What strikes me about that is… ← the reader's guide

It also carries the rules that make the output trustworthy rather than merely fluent:

• Never invent a ref, a quotation, or an attribution. If it was not retrieved, it is not available.
• Let the disagreement stand. Machloket is the Talmud's native form; collapsing three retrieved passages into one smooth answer is the commonest way to misrepresent it. If the sugya ends in teiku, say so.
• Say when the search failed. Some questions the index cannot serve — often the most interesting, because they are narrative rather than legal.
• Distinguish what the Talmud says from what the law is. The Gemara records positions, many rejected. For a practical question about your own life, it tells you to ask your rabbi.
• Know the tool's failure modes — §2 of RABBI.md names three we measured: homonyms (a question about talents returned a passage about a unit of weight), counterfactual grammar, and single-word lexical hooks.

Swap the persona

RABBI.md is one voice. The retrieval is indifferent to it — write SCHOLAR.md, CHAVRUTA.md, SKEPTIC.md and point the agent at that instead. The instructions worth keeping in any of them are the honesty rules above, not the personality.

Direct model use

from sentence_transformers import SentenceTransformer
m = SentenceTransformer("RobBobin/torah-embed")
q = m.encode("Represent this sentence for searching relevant passages: "
             "Can I say my prayers in the bathroom?", normalize_embeddings=True)
docs = m.encode(["<talmud segment>", "..."], normalize_embeddings=True)

Queries take the BGE instruction prefix; passages do not.

What this is not

• Absolute performance is modest. Conceptual questions reach MRR 0.116 and R@1 0.067 — the right sugya ranks first about seven times in a hundred. A measurable improvement on a hard task, not a solved problem.
• One epoch, batch 16, constrained by 16 GB of unified memory; the method prefers 64+. Further gain most likely comes from hardware, not method.
• It does not generalise to narrative or theological questions. On 26 hand-written questions about biblical narrative — the only query set here not generated from its own answer — fine-tuning changed 21 of 26 top hits and the change is not a clear improvement. One regression is instructive: asked "if Abraham had gone through with it, would Ishmael have inherited?", the model matched the counterfactual form of the question rather than its content. Training anchors came from legal rulings and can only ever yield legal questions. See RESULTS.md and paper/questions.pdf.
• Test questions were generated from their own rulings, so the relative comparison is sound while the absolute level is an optimistic bound.

Contents

Four designs this work eliminated

1. Dense-only retrieval — lexical and dense are complementary (top-10 overlap 63%, union R@10 0.836 vs ~0.73 each).
2. Sugya-level indexing — a ten-point apparent gain that reverses sign under a candidate-pool control.
3. Ruling-anchored training — the benchmark task is not the deployed task.
4. Exact-segment evaluation — scores a capability the product does not provide, understating dense retrieval by roughly half.

Licence and attribution

cc-by-nc-4.0, inherited from the source texts. Non-commercial use only.

• Talmud English: William Davidson Edition, translation by Rabbi Adin Even-Israel Steinsaltz, CC-BY-NC, via Sefaria.
• Mishneh Torah English: translation by Eliyahu Touger, CC-BY-NC, via Sefaria.
• Link data and structure: Sefaria.

Related

RobBobin/math-embed — the prior work this method adapts: knowledge-graph-guided contrastive fine-tuning for mathematical retrieval.

---

Robin Langer, with Claude (Anthropic) running the experiments.