clip-arch-classifier

clip-arch-classifier

Architectural style image classifier built on frozen CLIP ViT-B/32 embeddings.

Classifies exterior building photographs into 26 architectural styles. The classifier is a LinearSVC fitted on 512-dim L2-normalised CLIP image embeddings, with a Platt calibrator (logistic regression) on top to produce interpretable probabilities.

---

Model description

Files

---

Training data

Trained on the Architectural Styles Dataset (Curated and Extended): 9,767 images across 26 classes, split 70/15/15 train/val/test (stratified, seed 42).

The 26 classes are: Achaemenid, American Craftsman, American Foursquare, Ancient Egyptian, Art Deco, Art Nouveau, Baroque, Bauhaus, Beaux-Arts, Brutalism, Byzantine, Chicago school, Colonial, Deconstructivism, Edwardian, Georgian, Gothic, Greek Revival, International style, Novelty, Palladian, Postmodern, Queen Anne, Romanesque, Russian Revival, Tudor Revival.

---

Evaluation

Test set: 1,489 images (held-out, never seen during training or calibration)

Per-class F1 (test set)

Most-confused pairs

---

Intended use

• Classifying exterior building photographs by architectural style
• Educational and research use in architectural history and computer vision
• Input to downstream retrieval or recommendation systems

Not intended for:

• Interior photographs, architectural renders, or drawings
• Styles not in the 26-class vocabulary
• High-stakes decisions without human review

---

Limitations

• Weak classes: Edwardian (F1 = 0.53), Palladian (0.55), and International style (0.56) are the least reliable; treat their predictions as soft signals
• Style overlap: International ↔ Bauhaus and Postmodern ↔ International confusions reflect genuine art-historical ambiguity, not purely model error
• Geographic bias: training data is heavily Western/European
• Modality: trained exclusively on exterior photographs; performance on interiors and non-photographic images is undefined
• Leakage caveat: Ancient Egyptian and Novelty classes contain multiple photographs of the same landmark buildings; their F1 scores are likely slightly optimistic

---

Usage

import joblib
import torch
import torch.nn.functional as F
from PIL import Image
from transformers import CLIPModel, CLIPProcessor
from huggingface_hub import hf_hub_download

REPO_ID = "axel-riben/clip-arch-classifier"

# Load CLIP
processor = CLIPProcessor.from_pretrained("openai/clip-vit-base-patch32")
clip      = CLIPModel.from_pretrained("openai/clip-vit-base-patch32").eval()

# Load classifier and calibrator
svc   = joblib.load(hf_hub_download(REPO_ID, "linearsvc.joblib"))
platt = joblib.load(hf_hub_download(REPO_ID, "platt_calibrator.joblib"))

# Predict
image  = Image.open("building.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
    feats = clip.get_image_features(**inputs)
if not isinstance(feats, torch.Tensor):
    feats = feats.pooler_output
emb = F.normalize(feats, dim=-1).numpy()

scores = svc.decision_function(emb)          # (1, 26)
probs  = platt.predict_proba(scores)[0]      # (26,)

top5 = sorted(zip(platt.classes_, probs), key=lambda x: -x[1])[:5]
for label, prob in top5:
    print(f"{prob:.3f}  {label}")

---

Citation

If you use this model, please also cite the original dataset:

Danci, Marian Dumitru/dumitrux. (n.d.). Architectural Styles Dataset [Data set].
Kaggle. https://www.kaggle.com/datasets/dumitrux/architectural-styles-dataset

License

Code and model weights: MIT. Training data licences: see the dataset card.