Downloads · 30 days
35
18% of all-time downloads
balrampandey/indian-tax-law-slm-base
indian-tax-law-slm-base is a text generation model from balrampandey. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
<p align="center" <img src="https://img.shields.io/badge/params-31.8M-blue" / <img src="https://img.shields.io/badge/architecture-Llama-green" / <img src="https://img.shields.io/badge/domain-Indian%20Legal%20%26%20Tax…
Downloads · 30 days
35
18% of all-time downloads
All-time downloads
199
Public
Parameters
31.8M
127 MB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors127 MB · 99%
From the Hugging Face model README
indian-tax-law-slm-base is a domain-specialized language model for Indian legal and tax text, trained from scratch on a curated corpus of statutes, judgments, and legal documents. It is the first release in an ongoing series of increasingly capable open models targeting the Indian legal domain.
This initial model is a compact 31.8M-parameter base checkpoint that demonstrates strong, measurable domain specialization. Larger models — with substantially improved fluency and retrieval-grounded factual accuracy — are actively in development.
AutoModelForCausalLM, no custom code| Architecture | LlamaForCausalLM (grouped-query attention, RoPE, SwiGLU) |
| Parameters | 31.8M |
| Layers | 8 |
| Hidden size | 512 |
| Attention heads | 8 (4 KV heads) |
| Context length | 512 tokens |
| Vocabulary | 16,000 (custom byte-level BPE) |
| Precision | fp16 |
| Corpus | ~430M tokens, ~68% Indian legal by weight |
Held-out perplexity (lower = better):
| Domain | Perplexity |
|---|---|
| Indian legal | 40.7 |
| General English | 431.7 |
The model is 10.6× more confident on Indian legal text than on general text — clear, quantitative evidence of domain specialization. This is the intended behavior of a domain base model: it internalizes the vocabulary, structure, and register of Indian statutory and case law.
An interactive demo is available as a Hugging Face Space:
<gradio-app src="https://balrampandey-indian-tax-law-slm-demo.hf.space"></gradio-app>
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("balrampandey/indian-tax-law-slm-base")
tok = AutoTokenizer.from_pretrained("balrampandey/indian-tax-law-slm-base")
ids = tok("The assessee", return_tensors="pt")
out = model.generate(**ids, max_new_tokens=40, repetition_penalty=1.3)
print(tok.decode(out[0], skip_special_tokens=True))
The model is designed as a base for adaptation. Attach a LoRA adapter with peft:
from peft import LoraConfig, get_peft_model
model = get_peft_model(model, LoraConfig(
task_type="CAUSAL_LM", r=8, lora_alpha=16,
target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
))
A curated mix of general English (for language competence) and open Indian legal text (for domain specialization):
| Source | Content | Licence |
|---|---|---|
| allenai/c4 | General English | ODC-By |
| KanoonGPT/indian-legal-documents | Legal documents | Apache-2.0 |
| KanoonGPT/indian-case-laws | Case law | Apache-2.0 |
| 169Pi/indian_law | Legal Q&A / reasoning | Apache-2.0 |
| vihaannnn/Indian-Supreme-Court-Judgements-Chunked | SC judgments | MIT |
| varshith7/income-tax-act-india | Income Tax Act | CC-BY-4.0 |
The underlying text of Acts and judgments is exempt from copyright under s.52(1)(q) of the Indian Copyright Act; dataset compilations retain the licences above.
As an early, compact checkpoint, generation fluency is limited and outputs should not be treated as authoritative legal or tax information. Factual grounding via retrieval is planned for larger releases.
This model is the first step in a staged program:
Apache-2.0 (code and weights).