Downloads · 30 days
7
3% of all-time downloads
IndranilB/xtremedistil-l6-h384-uncased-legal-classification-hpt
xtremedistil-l6-h384-uncased-legal-classification-hpt is a text classification model from IndranilB. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.
This model is a fine-tuned version of microsoft/xtremedistil-l6-h384-uncased for multi-class legal document classification.
Downloads · 30 days
7
3% of all-time downloads
All-time downloads
234
Public
Parameters
22.7M
545 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors90.9 MB · 99%
From the Hugging Face model README
This model is a fine-tuned version of microsoft/xtremedistil-l6-h384-uncased for multi-class legal document classification.
It classifies European Union legal documents into 29 categories using the document text as input.
| Property | Value |
|---|---|
| Base Model | microsoft/xtremedistil-l6-h384-uncased |
| Architecture | BERT Encoder |
| Task | Multi-class Text Classification |
| Number of Classes | 29 |
| Framework | Hugging Face Transformers |
| Fine-tuning | Full Fine-tuning |
The model is designed for automatic categorization of legal documents.
Example applications include:
The model was trained on a corpus of European Union legal documents consisting of 29 document categories.
Examples include:
The dataset was split using an 80/20 train-validation split with a fixed random seed.
| Hyperparameter | Value |
|---|---|
| Epochs | 5 |
| Learning Rate | 5e-5 |
| Batch Size | 16 |
| Gradient Accumulation | 1 |
| Mixed Precision | BF16 |
| Optimizer | AdamW |
| Evaluation Strategy | Every Epoch |
| Checkpoint Strategy | Every Epoch |
| Early Stopping | Patience = 2 |
| Best Model Selection | Validation Accuracy |
The original tokenizer from the base model was used.
DataCollatorWithPadding| Metric | Score |
|---|---|
| Validation Accuracy | 68.59% |
| Micro F1 | 68.59% |
| Validation Loss | 2.1866 |
from transformers import AutoTokenizer
from transformers import AutoModelForSequenceClassification
import torch
model_name = "IndranilB/xtremedistil-l6-h384-uncased-legal-classification-hpt"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
text = "Your legal document goes here."
inputs = tokenizer(
text,
truncation=True,
return_tensors="pt",
)
with torch.no_grad():
outputs = model(**inputs)
prediction = outputs.logits.argmax(dim=-1).item()
print(prediction)
The model was trained using the Hugging Face Trainer API with:
Future improvements include:
This work builds upon: