lebse

LeBSE — Legal-domain Sentence Embeddings (a legal-adapted LaBSE)

LeBSE-v2 is LaBSE fine-tuned with citation supervision on U.S. case law, so legally-related opinions land closer together. It is a drop-in sentence-transformers model (768-dim, same tokenizer, keeps LaBSE's multilingual base).

from sentence_transformers import SentenceTransformer
m = SentenceTransformer("ahbond/lebse")
emb = m.encode(["The district court lacked subject-matter jurisdiction over the claim."],
               normalize_embeddings=True)

How it was trained

SPECTER-style contrastive fine-tuning: positive pairs are (citing opinion body, cited opinion body) from the CourtListener citation graph (100,344 pairs); the rest of the batch are negatives (MultipleNegativesRankingLoss). One NVIDIA A10, batch 96, max_seq_length 128, 2 epochs. No case outcomes are used — the signal is citation relatedness only.

Evaluation (held out, opinion-disjoint)

LeBSE-v2 dramatically improves the relatedness it was trained on and transfers a small-but-significant amount to an independent legal relation (a district opinion ↔ its appellate reviewer, matched by docket number, never trained on). It also improves embedding isotropy (anisotropy 0.570 → 0.259).

An earlier v1 used unsupervised SimCSE and did not beat base LaBSE — see the repo for that honest negative result. This model is v2.

Intended use & limits

Legal opinion/paragraph retrieval, citation recommendation, clustering, and as a frozen legal-domain feature extractor. Not for legal advice or case outcome decisions. U.S. federal law only; specialized to citation-type relatedness; encodes a ~128-token paragraph, not a whole opinion.

Code, training, and evaluation: https://github.com/ahb-sjsu/lebse · License: Apache-2.0 (training data is U.S. federal case law, public domain).