saul-7b-indian-law-lora

Saul-7B Indian Law LoRA

A LoRA fine-tune of Equall/Saul-7B-Instruct-v1 for Indian legal question answering.

Model

Base model: Equall/Saul-7B-Instruct-v1

Fine-tuned model: Demondiablo/saul-7b-indian-law-lora

This repository contains the LoRA adapter and tokenizer files required to load the fine-tuned model on top of the base Saul-7B model.

Training Data

The fine-tuning dataset was constructed from three Indian legal datasets:

1. kaushik-harsh-99/Indian-legal-data-v2
2. GSMS-B/Indian-Legal-QA-BNS-BNSS-BSA
3. nisaar/Articles_Constitution_3300_Instruction_Set

The combined dataset contained 21,665 examples after preprocessing.

Dataset 1 was limited to 12,000 examples.

Evaluation

The models were evaluated on a fixed 434-question held-out test set.

The evaluation set contained:

• 254 examples from Indian-legal-data-v2
• 120 examples from BNS-BNSS-BSA
• 60 examples from Constitution-Articles

Both the base model and fine-tuned model were evaluated on the same questions and reference answers.

Gemini LLM-as-a-Judge Results

The final evaluation used Gemini 3.5 Flash Lite with a structured 1–5 scoring rubric.

For the hallucination metric, 5 indicates no hallucination and 1 indicates severe hallucination.

Question-Level Comparison

• Fine-tuned wins: 130 / 434 (30.0%)
• Base wins: 94 / 434 (21.7%)
• Ties: 210 / 434 (48.4%)

Evaluation Artifacts

The complete evaluation record is available under:

evaluation_backup/

including:

• heldout_434_legal_test_set.csv
• base_predictions_checkpoint.json
• fine_tuned_predictions_checkpoint.json
• saul_legal_eval_results.csv
• saul_legal_eval_results.json
• saul_legal_llm_judge_results.csv
• gemini_evaluation_summary.json

These files preserve the held-out test set, model predictions, question-level comparisons, and final Gemini evaluation results.

Intended Use

This model is intended for research, experimentation, and educational use in Indian legal NLP.

It should not be treated as a substitute for qualified legal advice, professional legal research, or authoritative legal sources. Model outputs may contain factual, legal, or interpretive errors.

Files

The main model repository contains the LoRA adapter and tokenizer artifacts:

adapter_model.safetensors
adapter_config.json
tokenizer.json
tokenizer.model
tokenizer_config.json
chat_template.jinja

The base model weights are not included in this repository and must be obtained from:

Equall/Saul-7B-Instruct-v1

Tools

Built using:

• Unsloth
• Hugging Face Transformers
• PEFT / LoRA
• Hugging Face Datasets
• Gemini LLM-as-a-judge evaluation

Status

Fine-tuning and evaluation completed.

Final evaluation coverage: 434 / 434 held-out questions.