PatentSBERTa-green-classifier

PatentSBERTa — Green Patent Classifier

A fine-tuned AI-Growth-Lab/PatentSBERTa model for binary classification of patent claims as green technology (1) or not green (0).

Developed as part of the Applied Deep Learning (AAU, Spring 2025) exam assignment on active learning, human-in-the-loop labelling, and multi-agent systems for patent classification.

Model Details

Training

Pipeline overview

1. Part A–B: Frozen PatentSBERTa baseline + uncertainty-based active-learning pool selection
2. Part C: QLoRA-tuned Llama-3.1-8B powering a LangGraph Multi-Agent System (Advocate → Skeptic → Judge → Exception) to generate silver labels on the 15k most uncertain patents
3. Part D: Human-in-the-loop review of 100 critical samples → gold labels, then final full-parameter fine-tuning of PatentSBERTa

Training data

Hyperparameters

Hardware

• 4 × NVIDIA L4 (24 GB each), DDP via torchrun
• AAU AI-Lab (SLURM cluster)
• Wall-clock time: ~23 minutes

Evaluation

Evaluated on the held-out eval_silver split (10,000 samples, balanced).

Confusion Matrix

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "CTB2001/PatentSBERTa-green-classifier"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

claim = "A wind turbine blade comprising a spar cap formed from pultruded carbon strips..."
inputs = tokenizer(claim, return_tensors="pt", truncation=True, max_length=512)

with torch.no_grad():
    logits = model(**inputs).logits
    pred = torch.argmax(logits, dim=-1).item()

print("green" if pred == 1 else "not-green")

Intended Use

• Primary: Classifying patent claims as green / not-green technology
• Domain: Patent text (US/EP/WO first claims)
• Not suitable for: General-purpose NLI, legal advice, or production patent screening without additional validation

Limitations

• Trained and evaluated on silver labels (machine-generated); a small fraction may be noisy
• Only 100 gold (human-verified) labels were available — upsampled 25× to amplify signal
• Performance on out-of-domain patent offices or languages is unknown

Citation

@misc{trost-bertelsen2025patentsberta-green,
  author       = {Trøst-Bertelsen, Christian},
  title        = {PatentSBERTa Green Patent Classifier},
  year         = {2025},
  howpublished = {Hugging Face Model Hub},
  url          = {https://huggingface.co/CTB2001/PatentSBERTa-green-classifier}
}

Author

Christian Trøst-Bertelsen — Aalborg University, Student ID 20224083 Course: Applied Deep Learning, 8th semester, Spring 2025