genai-smartcity-classifier

GenAI Smart City Classifier (DeBERTa v3 Base Fine-Tune)

Binary transformer classifier detecting whether a text describes a Generative AI (GenAI) application in a smart city context.

The full codebase can be found here.

Labels

• 0: GenAI used for smart city application
• 1: Not related

id2label = {0: "GenAI used for smart city application", 1: "Not related"}

Model Card Summary

• Base: microsoft/deberta-v3-base
• Tokenizer: DebertaV2Tokenizer (same as base)
• Max length used in training batches: 512 (inference examples use 256)
• Loss: Custom focal loss (γ=2) + label smoothing (0.1)
• Scheduler: Cosine, warmup 10%
• Epochs: 8, batch size 8 (train) / 16 (eval)
• Calibration: Temperature scaling (optimal ≈ 0.602)

Quick Start

import torch
from transformers import DebertaV2Tokenizer, AutoModelForSequenceClassification

MODEL_ID = "joaocarlosnb/genai-smartcity-classifier"  # replace with actual repo id
TEMP = 0.602
id2label = {0: "GenAI used for smart city application", 1: "Not related"}

tokenizer = DebertaV2Tokenizer.from_pretrained(MODEL_ID)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID)
model.eval()

def predict(text, max_length=256, apply_temp=True):
    inputs = tokenizer(text, truncation=True, padding="max_length",
                       max_length=max_length, return_tensors="pt")
    with torch.no_grad():
        logits = model(**inputs).logits
        if apply_temp:
            logits = logits / TEMP
        probs = torch.softmax(logits, dim=-1)[0]
    top = int(probs.argmax())
    return {
        "label": id2label[top],
        "probabilities": {id2label[i]: float(p) for i, p in enumerate(probs)}
    }

print(predict("We apply a diffusion model to simulate traffic for urban planning."))

Installation

pip install transformers torch

Input Guidance

Short technical sentences or abstract fragments (English). Truncate >512 tokens automatically.

Limitations

• Binary only (no “mentioned, not used” middle class)
• English academic / technical domain bias
• Not evaluated for adversarial or multilingual robustness

Intended Use

Research, corpus analysis, and exploratory filtering. Human review is recommended before operational deployment.

Dataset

Training data hosted separately (same namespace). Contains augmented, adaptive, contrastive, and diagnostic subsets.

Reproducibility Notes

Set seed=42. Use DebertaV2Tokenizer with max_length=512 for full retraining.

Citation (Placeholder)

Bittencourt, J. C. N., Flores, T. K. S., Jesus, T. C., & Costa, D. G. (2025). On the Role of AI in Building Generative Urban Intelligence. In Review. https://doi.org/10.21203/rs.3.rs-7131966/v1

License

See repository LICENSE (ensure compatibility with upstream model license).

Security

Do not hard-code Hugging Face tokens. Use environment variable: `export