contractnli-distilbert-nda

contractnli-distilbert-nda

DistilBERT fine-tuned on ContractNLI — fastest inference, recommended for production

Task

Document-level NLI for Non-Disclosure Agreements (NDAs)

Given an NDA contract and a hypothesis about a standard provision, classify as:

• Entailment: The provision is present in the contract
• Contradiction: The provision is explicitly excluded
• NotMentioned: The contract does not address this provision

Performance

Comparison

17 NDA Provisions Checked

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "Agreemind/contractnli-distilbert-nda"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

hypothesis = "All Confidential Information shall be expressly identified by the Disclosing Party."
premise = "Section 2.1: Any information disclosed must be marked as Confidential..."

inputs = tokenizer(hypothesis, premise, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
    logits = model(**inputs).logits
    probs = torch.softmax(logits, dim=-1)
    pred = ["Entailment", "Contradiction", "NotMentioned"][probs.argmax()]
    print(f"Prediction: {pred} (confidence: {probs.max():.3f})")

Training Details

• Dataset: ContractNLI (607 NDAs, 17 hypotheses)
• Train/Dev/Test: 423/61/123 documents → 33,974/5,131/9,373 span-level examples
• Base model: distilbert-base-uncased
• Loss: Weighted Cross-Entropy
• Learning rate: 3e-5 (aligned with ContractNLI paper)
• Epochs: 5 with early stopping (patience=3)
• Batch size: 8
• Max sequence length: 512

Citation

@inproceedings{koreeda-manning-2021-contractnli,
    title = "ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts",
    author = "Koreeda, Yuta and Manning, Christopher",
    booktitle = "Findings of EMNLP 2021",
    year = "2021",
}

License

MIT