cuad-clause-risk-cnn

Clause risk detection — TextCNN

Multi-label classifier flagging six risk-relevant clause types in commercial contracts, fine-tuned on CUAD v1.

Source and training code: https://github.com/ManasDasri/NNDL

Labels

Six independent sigmoid outputs, not a softmax: contracts routinely carry several of these clause types at once.

Results

Held-out test split, split by contract so no document appears in training.

Chunk-level macro F1: 0.663 Document-level macro F1: 0.840 (max-pooled over windows)

How it was trained

• Contracts are split into overlapping windows of 300 words with 100 words of overlap, because 97% of CUAD contracts exceed a 512-token context and 70% exceed 4,096.
• A window takes a label when it meaningfully overlaps an annotated clause span.
• Loss is BCE weighted by each label's negative-to-positive ratio, which runs 22x to 68x. 85% of windows carry no label at all, so an unweighted loss collapses to predicting nothing.
• Splits are assigned per contract, never per window.

Thresholds

Decision thresholds are tuned per label on the validation split rather than fixed at 0.5, because each label has its own positive rate. Using 0.5 will cost you recall.

Usage

This is a TextCNN, not a transformers architecture, so there is no from_pretrained for it. Load it with the training repository:

from legal_risk_classifier.runtime import CNNRuntime

runtime = CNNRuntime('path/to/this/download')
prediction = runtime.predict_document(contract_text)
print(prediction.predicted, prediction.document_scores)

Limitations

• Trained on 358 contracts. The rarest label, Non-Compete, appears in around 1.5% of windows and is correspondingly the weakest.
• Run-to-run variance is about ±0.03 macro F1 on identical settings, so small differences between checkpoints are not meaningful.
• CUAD is US commercial contracts. Behaviour on other jurisdictions or contract families is untested.
• This is a research artifact and not legal advice. It is a triage aid for locating clauses a lawyer should read, not a substitute for reading them.