cuad-cross-encoder-v10

cuad-cross-encoder-v10

A cross-encoder reranker fine-tuned for legal clause retrieval in contract review workflows. Built on top of cross-encoder/ms-marco-MiniLM-L-6-v2 and fine-tuned on a combination of CUAD, ACCORD, LEDGAR, ContractNLI, and EDGAR-sourced contract pairs.

Deployed as ONNX INT8 for in-browser inference via WebAssembly / ONNX Runtime Web.

---

Intended Use

• Primary: Reranking retrieved contract chunks against natural-language clause queries (e.g. "What are the governing law provisions?", "What IP does each party retain?")
• Domains covered: Joint Venture, Intellectual Property, Non-Compete / Non-Solicit, Non-Disclosure Agreement (NDA)
• Not intended for: General-purpose document retrieval, non-legal domains, or as a standalone legal advisor

---

Training Data

Total: ~48,268 training pairs · 5,612 validation pairs

Pairs are (query, positive_chunk, negative_chunk) triplets. Negatives are a mix of hard negatives (wrong clause from same contract) and random negatives (chunks from other contracts).

---

Training Details

---

Evaluation

Evaluated on a held-out set of 16 contracts across 4 clause domains. Each contract is queried with 3–8 clause-type questions; the top-ranked chunk is scored as pass (correct clause returned), partial (correct section but wrong chunk boundary), or fail.

Test contracts (JV): MightyCell Batteries, BorrowMoney.com, Galera Therapeutics, MINDA IMPCO Technologies, Kiromic Biopharma, Novo Integrated Sciences, Transphorm / Aizu Fujitsu, Valence Technology / Baoding Fengfan, Veoneer

Test contracts (IP): Armstrong Flooring, Cerence Inc, Garrett Motion, Rare Element Resources

Test contracts (NDA): Kite Pharma / Gilead Sciences, Fortune Brands / Norcraft Companies, Aspect Medical Systems / Tyco Healthcare

---

Usage

ONNX Runtime (recommended for browser / edge)

import onnxruntime as ort
from transformers import AutoTokenizer
import numpy as np

tokenizer = AutoTokenizer.from_pretrained("datgacon/cuad-cross-encoder-v10")
session = ort.InferenceSession("model_quantized.onnx")

query = "What governing law applies to this agreement?"
passage = "This Agreement shall be governed by and construed in accordance with the laws of the State of Delaware."

inputs = tokenizer(query, passage, return_tensors="np", max_length=512, truncation=True, padding=True)
outputs = session.run(None, {k: v for k, v in inputs.items() if k in ["input_ids", "attention_mask", "token_type_ids"]})
score = outputs[0][0][0]
print(f"Relevance score: {score:.4f}")

sentence-transformers (PyTorch)

from sentence_transformers.cross_encoder import CrossEncoder

model = CrossEncoder("datgacon/cuad-cross-encoder-v10")

query = "What governing law applies to this agreement?"
passages = [
    "This Agreement shall be governed by the laws of the State of Delaware.",
    "Each party shall maintain the confidentiality of the other party's information.",
    "The term of this Agreement shall commence on the Effective Date.",
]

scores = model.predict([(query, p) for p in passages])
ranked = sorted(zip(scores, passages), reverse=True)
for score, passage in ranked:
    print(f"{score:.4f}  {passage[:80]}")

---

Limitations

• Trained on US commercial contracts (CUAD corpus); may underperform on EU, UK, or public-sector agreements
• Partial matches are common at clause-boundary edges — chunk size and overlap in the retrieval pipeline significantly affect results
• Not a legal advisor — scores indicate retrieval relevance, not legal interpretation
• Performance on clause types outside the four trained domains (JV, IP, NC, NDA) is untested

---

Citation

If you use this model, please cite the underlying datasets:

@article{hendrycks2021cuad,
  title={CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review},
  author={Hendrycks, Dan and others},
  journal={arXiv preprint arXiv:2103.06268},
  year={2021}
}