Legalbert_Finetuned

🧠 LegalBERT Finetuned — Contract Clause Classification

📜 Model Overview

LegalBERT_Finetuned is a domain-specific transformer model fine-tuned for legal clause classification and tier-based contract review.
This model forms the backbone of the NLP Contract Summarization & Tier-wise Clause Review project — an AI-driven framework that automates clause identification, categorization, and prioritization for large-scale legal documents.

• Base model: nlpaueb/legal-bert-base-uncased
• Task: Multi-class text classification (Clause → Label → Tier)
• Domain: Legal NLP (Contractual Clauses)
• Language: English
• License: MIT
• Author: Bhargav Bidkar

---

🚀 Intended Use

This model is designed for:

• Automatic clause-type classification within contracts and legal documents.
• Assigning review tiers (1–5) based on the criticality and legal importance of each clause.
• Serving as a preprocessing or intelligence layer in AI-assisted legal review pipelines.

✅ Suitable Applications

• Contract review automation
• Clause categorization and summarization
• Legal document analytics
• Risk identification and prioritization

⚠️ Limitations / Misuse

• The model is not a legal advisor and should not replace human legal judgment.
• Trained primarily on commercial contracts from the CUAD dataset — performance may degrade on other document types (e.g., litigation, IP filings).
• Sensitive to domain drift; fine-tuning may be required for other jurisdictions or contract formats.

---

📚 Training Data

Model fine-tuned on the Contract Understanding Atticus Dataset (CUAD):

• 510 commercial contracts
• ~13,000 annotated clauses
• 41 distinct clause categories (Termination, Liability, IP Ownership, etc.)

Each clause was mapped to one of five review tiers:

---

⚙️ Training Details

• Base model: nlpaueb/legal-bert-base-uncased
• Optimizer: AdamW
• Learning Rate: 2e-5
• Batch Size: 16
• Epochs: 3–5
• Loss Function: CrossEntropyLoss
• Hardware: NVIDIA A100 / RTX GPU

Evaluation

The model demonstrated strong generalization across unseen clause structures and consistent labeling across the 41 categories.

---

🧠 Model Architecture

1. Clause Classification

   • Input: Raw contract clause text
   • Output: Clause Label + Tier
   • Model: Fine-tuned LegalBERT

2. Summarization (Downstream)

   • Integrated with GPT-4.1-mini for abstractive contract summaries.
   • Not part of this specific model checkpoint but used in the full pipeline.

---

🧮 Preprocessing

• Extracted clauses from PDF/DOCX using pdfplumber and python-docx.
• Cleaned text: removed numbering, headers, and redundant whitespace.
• Consolidated all clause CSVs into combined_clauses.csv.
• Normalized and standardized labels from the CUAD dataset.
• Mapped labels to tier levels (1–5) based on contractual criticality.

---

📊 Example Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("bhargav-07-bidkar/LegalBERT_Finetuned")
model = AutoModelForSequenceClassification.from_pretrained("bhargav-07-bidkar/LegalBERT_Finetuned")

text = "The Company shall indenify and hold the Client harmless from any damages or liabilities."
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
outputs = model(**inputs)
predicted_label = torch.argmax(outputs.logits, dim=1).item()
print(predicted_label)