Fine-Tunned-GPT

Model Card for Indian Penal Code Section Classifier

<!-- Provide a quick summary of what the model is/does. -->

This model is a fine-tuned version of unsloth/llama-3.2-3b-instruct-bnb-4bit specifically designed for text classification to identify Indian Penal Code (IPC) sections relevant to legal text inputs. It is trained to understand legal queries and classify them under appropriate IPC sections.

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

This model is developed as part of a project focused on fine-tuning large language models for question-answering in specialized domains, particularly legal and medical contexts. This specific model focuses on the legal domain and is trained to classify text inputs, such as legal questions or descriptions of scenarios, into categories corresponding to sections of the Indian Penal Code. The model leverages the llama-3.2-3b-instruct-bnb-4bit base model and is fine-tuned using PEFT (Parameter-Efficient Fine-Tuning) techniques for efficient adaptation to the specific task.

• Developed by: Nehal , Nilanshi and Ayush
• Funded by: [More Information Needed, Optional]
• Shared by: [More Information Needed, Optional]
• Model type: Text Classification Language Model
• Language(s) (NLP): English
• License: [More Information Needed - Specify the license, e.g., Apache 2.0, CC BY-SA 4.0. Choose a license appropriate for your use case and dataset.]
• Finetuned from model: unsloth/llama-3.2-3b-instruct-bnb-4bit

Model Sources

<!-- Provide the basic links for the model. -->

• Repository: [Your GitHub Repository Link - Replace with the actual link to your GitHub repo: e.g., https://github.com/your-username/your-repo-name]
• Paper: [More Information Needed, Optional - If there's a related paper, link it here]
• Demo: [More Information Needed, Optional - If you have a demo, link it here]

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

The model is intended for direct use in applications that require automated classification of legal text or queries into Indian Penal Code sections. Users can input legal questions, case descriptions, or textual scenarios, and the model will output the predicted IPC section(s) that are most relevant. This can be used for preliminary legal analysis, information retrieval, and organizing legal documents.

Downstream Use

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

This model can be integrated into larger systems and applications, such as:

• Legal Assistants: As a component in AI-powered legal assistant tools to help legal professionals quickly identify relevant IPC sections.
• Domain-Specific Chatbots: To enhance legal chatbots by enabling them to understand and categorize user queries based on IPC sections.
• Research Assistance: To aid legal researchers in quickly categorizing and analyzing legal texts based on IPC classifications.
• Educational Tools: For law students and educators to explore and understand the categorization of legal scenarios under the Indian Penal Code.

Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

This model should not be used for:

• Providing definitive legal advice. The model is a tool for classification and should not replace professional legal counsel.
• Automated legal decision-making without human oversight. The model's predictions should be reviewed and validated by legal professionals.
• Any application that could lead to unjust or discriminatory outcomes. The model's performance may vary, and biases in the training data could be reflected in its predictions.
• Use in high-stakes scenarios where incorrect classifications could have serious legal or financial consequences without proper validation and human review.

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

• Data Bias: The model's performance is dependent on the quality and representativeness of the training data. If the dataset contains biases (e.g., under-representation of certain types of legal cases or sections), the model may exhibit these biases in its predictions.
• Accuracy Limitations: As a fine-tuned language model, it may not always achieve perfect accuracy in classification. The complexity of legal language and nuances in case law can pose challenges.
• Over-reliance: Users should avoid over-relying on the model's output without critical evaluation and validation by legal experts.
• Evolving Legal Landscape: Legal statutes and interpretations can change over time. The model's knowledge is based on the data it was trained on and may not reflect the most up-to-date legal information.
• General LLM Limitations: Like other large language models, this model may occasionally generate incorrect or nonsensical outputs.

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases, and limitations of the model.

Recommendations for responsible use:

• Human Oversight: Always use the model's predictions as a starting point and ensure review and validation by qualified legal professionals.
• Contextual Understanding: Consider the model's output within the broader context of the legal situation and not in isolation.
• Continuous Evaluation: Regularly evaluate the model's performance and retrain or fine-tune it with updated data to maintain accuracy and relevance.
• Transparency: Be transparent with end-users about the model's capabilities and limitations when deploying it in applications.
• Ethical Considerations: Use the model ethically and responsibly, ensuring it does not contribute to unfair or discriminatory outcomes.

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoModelForSequenceClassification, AutoTokenizer
from peft import PeftModel, PeftConfig
import torch

model_name = "unsloth/llama-3.2-3b-instruct-bnb-4bit" # Replace with your actual model ID after pushing to hub
peft_model_id = "[Your Model ID on Hugging Face Hub]" # e.g., "ayush0504/IndianPenalCodeSections"

config = PeftConfig.from_pretrained(peft_model_id)
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
model = AutoModelForSequenceClassification.from_pretrained(config.base_model_name_or_path, num_labels= [Number of IPC Sections in your dataset],  device_map='auto') # Set num_labels correctly
model = PeftModel.from_pretrained(model, peft_model_id)
model.eval()

def classify_ipc_section(text):
    inputs = tokenizer(text, return_tensors="pt").to(model.device)
    with torch.no_grad():
        outputs = model(**inputs)
    predictions = outputs.logits.argmax(dim=-1) # Or apply softmax if needed
    predicted_section_index = predictions.item()
    # Map index to IPC Section label (you'll need to create this mapping)
    # predicted_section_label = ipc_section_mapping[predicted_section_index]
    return predicted_section_index # Or return predicted_section_label

# Example usage
text_query = "A person intentionally causes hurt to deter a public servant from discharge of his duty."
predicted_section = classify_ipc_section(text_query)
print(f"Predicted IPC Section Index: {predicted_section}") # Or print the label

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

The model was trained on the IndianPenalCodeSections Dataset dataset available on Hugging Face Datasets. This dataset comprises question-answer pairs related to Indian Penal Code sections. It includes questions or scenarios and corresponding IPC section labels representing the correct classification. The dataset was curated and preprocessed to remove noise and improve data quality for training.

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

The model was fine-tuned using Parameter-Efficient Fine-Tuning (PEFT) techniques, specifically likely using LoRA (Low-Rank Adaptation), although the specific PEFT method should be confirmed. The base model unsloth/llama-3.2-3b-instruct-bnb-4bit was adapted for the text classification task.

Preprocessing

Text data was preprocessed before training, including:

• Noise Removal: Removal of punctuation, special characters, and irrelevant text.
• Tokenization: Text was tokenized using the tokenizer associated with the base model (unsloth/llama-3.2-3b-instruct-bnb-4bit).
• Normalization: Text normalization steps were applied to ensure consistency (e.g., lowercasing, handling of special characters).
• Contextual Phrases: For legal questions, context phrases such as "according to section" or "in law" were added to some training examples to potentially improve model accuracy in understanding legal context.

Training Hyperparameters

• Training regime: Fine-tuning, likely with LoRA or similar PEFT method, quantization using BNB (BitsAndBytes) in 4-bit precision. [Specify if mixed precision or other details are known].

Speeds, Sizes, Times

[More Information Needed, Optional - Add information about training time, checkpoint size, throughput if available]

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Data, Factors & Metrics

Testing Data

The model was evaluated on a [Specify if same as training data or separate split] split of the IndianPenalCodeSections Dataset dataset.

Factors

[More Information Needed, Optional - Specify factors used for evaluation disaggregation, e.g., complexity of legal questions, specific types of IPC sections. If not applicable, state "Evaluation was performed on the overall dataset without disaggregation by specific factors."]

Metrics

The model's performance was evaluated using the following metrics:

• Cosine Similarity: Measures the semantic similarity between predicted and actual IPC section descriptions (if applicable, or if comparing embeddings).
• Jaccard Similarity: Measures the overlap of keywords between predicted and actual IPC section descriptions (if applicable, or if comparing token sets).
• Euclidean Distance: Measures the distance between vector representations of predicted and actual IPC section descriptions (if applicable, for embedding-based evaluation).
• Manhattan Distance: Measures the distance between vector representations of predicted and actual IPC section descriptions (if applicable, for embedding-based evaluation).
• Pearson Correlation: Assesses the linear relationship between vector representations of predicted and actual IPC section descriptions (if applicable, for embedding-based evaluation).
• Accuracy: [Crucially Important Metric for Classification] Report standard classification accuracy: the percentage of correctly classified IPC sections.

Results

[More Information Needed - This is crucial. Add your evaluation results here, especially Accuracy. Report accuracy and, if available and relevant, results for other metrics like Cosine Similarity, Jaccard Similarity etc., if you used them for evaluation. Provide a table or clear summary of the metrics on your test set.]

Summary

[More Information Needed - Summarize your evaluation results. Highlight the model's strengths and weaknesses based on the evaluation metrics.]

Model Examination

[More Information Needed, Optional - If you conducted any interpretability analysis or model examination, describe it here.]

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here -->

Carbon emissions during fine-tuning can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [More Information Needed - Specify the hardware used for training, e.g., GPU model]
• Hours used: [More Information Needed - Estimate the training hours]
• Cloud Provider: [More Information Needed - If using a cloud provider like AWS, GCP, Azure]
• Compute Region: [More Information Needed - Region where computation was performed]
• Carbon Emitted: [More Information Needed - Calculate using the ML Impact calculator if possible]

Technical Specifications

Model Architecture and Objective

[More Information Needed - Briefly describe the model architecture (Transformer-based) and its objective (text classification of IPC sections).]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed - Specify software versions, libraries used (e.g., Python, Transformers, PEFT, CUDA)]

Citation

[More Information Needed, Optional - If you want to be cited, provide citation information.]

BibTeX:

[More Information Needed -  BibTeX citation if applicable]

Citation

[More Information Needed, Optional - If you want to be cited, provide citation information.]

APA:

[More Information Needed - APA citation if applicable] Use code with caution.

Glossary

[More Information Needed, Optional - Define any specific terms relevant to your model or domain that might be helpful for users.]

More Information

[More Information Needed, Optional - Link to blog posts, project websites, or further documentation if available.]

Model Card Authors

[More Information Needed - List the authors of this model card.]

Model Card Contact

[More Information Needed - Provide contact information for questions about the model card or the model.]

Framework versions

• PEFT 0.14.0
• Transformers [Specify Transformers version used, e.g., 4.x.x]
• Python [Specify Python version used, e.g., 3.9.x]
• [Add other relevant framework versions, e.g., Torch, CUDA if applicable]