InLegalBERT-Appeal-Predictor-LoRA

Model Card for InLegalBERT-Appeal-Predictor-LoRA

Model Details

Model Description

This model is a fine-tuned version of law-ai/InLegalBERT designed to predict the outcome of legal appeals (Accepted vs. Rejected) based on the text of the legal judgment. It utilizes Low-Rank Adaptation (LoRA) for parameter-efficient fine-tuning (PEFT), making it highly efficient while retaining strong predictive capabilities on complex, domain-specific Indian legal texts.

To handle extremely long legal documents, this model utilizes a specialized "Head+Tail" truncation strategy, analyzing the first 128 tokens and the last 383 tokens of a document to capture both the introductory context and the final verdict rationale.

• Developed by: Swastik Sharma (swastik7805 / Beelzi)
• Model type: Sequence Classification (PEFT/LoRA)
• Language(s) (NLP): English (Indian Legal Context)
• Finetuned from model: law-ai/InLegalBERT

Uses

Direct Use

The model is intended for legal researchers, practitioners, and automated legal analysis pipelines to predict the likelihood of a legal appeal being accepted (Class 1) or rejected (Class 0) based on the textual content of the case.

Out-of-Scope Use

This model should not be used as a substitute for professional legal counsel. The predictions are statistically generated based on historical data and do not guarantee actual future court outcomes.

How to Get Started with the Model

Use the code below to get started with the model. Because the model was trained using a specific Head+Tail truncation strategy for texts exceeding 512 tokens, it is highly recommended to apply the same truncation during inference:

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
from peft import PeftModel

base_model_name = "law-ai/InLegalBERT"
peft_model_id = "Beelzi/InLegalBERT-Appeal-Predictor-LoRA"

tokenizer = AutoTokenizer.from_pretrained(peft_model_id)
base_model = AutoModelForSequenceClassification.from_pretrained(base_model_name, num_labels=2)
model = PeftModel.from_pretrained(base_model, peft_model_id)

def predict_long_text(text):
    tokens = tokenizer(text, truncation=False, return_tensors="pt")
    input_ids = tokens["input_ids"]
    attention_mask = tokens["attention_mask"]

    if input_ids.shape[1] > 512:
        head_ids = input_ids[:, :128]
        head_mask = attention_mask[:, :128]
        tail_ids = input_ids[:, -383:]
        tail_mask = attention_mask[:, -383:]
        
        sep_id = torch.tensor([[tokenizer.sep_token_id]], device=input_ids.device)
        sep_mask = torch.tensor([[1]], device=attention_mask.device)
        
        input_ids = torch.cat([head_ids, tail_ids, sep_id], dim=1)
        attention_mask = torch.cat([head_mask, tail_mask, sep_mask], dim=1)

    input_ids = input_ids.to(model.device)
    attention_mask = attention_mask.to(model.device)

    model.eval()
    with torch.no_grad():
        outputs = model(input_ids=input_ids, attention_mask=attention_mask)
        logits = outputs.logits
        final_pred = torch.argmax(logits, dim=1).item()

    return "Accepted" if final_pred == 1 else "Rejected"