aidetect-vit-b16

AI Image Detector — ViT-B/16 (CLIP-initialised)

Binary classifier: real photograph vs AI-generated image. Fine-tuned from the openai/clip-vit-base-patch16 vision tower on ~195,000 images spanning 1,100+ distinct generators, and — the part that usually goes unreported — measured on generators it never saw during training.

🔍 Try it in the browser

Why another detector

Published AI-image detectors routinely report 99% and then collapse on anything their training generator did not produce. Wang et al. (2020) documented it; Bird & Lotfi (2024) hit 92.98% on CIFAKE using a single generator and named that as their central limitation. This model is an attempt to answer the question their limitation raises: how far does accuracy actually fall on generators nobody trained against?

The answer here is 97.6% → ~89.8%, and the failures are not where intuition says.

Results

Two notes on reading this table honestly:

• The 0.898 is at the deployed threshold of 0.71, which was fitted on one half of the unseen-generator split and reported on the held-out half. At the threshold fitted on in-distribution validation instead, the same model scores 0.881 there. The threshold is part of the model; both numbers are given so neither looks cherry-picked.
• reddit/test is 83% fake, so read its AUROC rather than its accuracy — a model that simply answers "always AI" scores 82.6% on it.

Architecture comparison

A ResNet-50 trained on the identical data (2 epochs, fixed budget) reaches 0.945 in-distribution but only 0.813 / AUROC 0.896 on unseen generators. The gap between the two architectures triples once the generator is unfamiliar — which is the argument for CLIP initialisation, arriving as a measurement rather than a claim. Two caveats, so the comparison is read for what it is: the ResNet had 2 epochs against the ViT's 5, and it ran under CLIP normalisation rather than ImageNet normalisation, since the pipeline is shared across every model. Both mildly handicap it — though neither explains why the AUROC gap is far wider out of distribution (6.4 points) than in it (0.8).

Unseen generators are mostly caught. On core/test: gpt-image-1.5 0.962, z-image-turbo 0.952, flux.2-klein-9b 0.905 — none of them in training. Transfer is not uniform: midjourney-7 (0.856) and recraft-v3 (0.600) are the weak points. The residual error is dominated by the other direction: genuine photographs from unfamiliar camera pipelines (DOCCI, ImageNet) being called AI. Full-range resolution augmentation cut that false-positive rate from 27.3% to 17.5%, and refitting the decision threshold on out-of-distribution data (0.38 → 0.71) took it to 9.5%.

What is in this repository

Each run changes exactly one thing, so any difference between them is attributable. runC is the model served by the demo and the one these numbers describe.

runD is kept because a negative result is still a result: adding a simulated phone camera pipeline made the detector worse at catching fakes and worse at leaving real photographs alone. Each folder holds checkpoints/best.pt, checkpoints/last.pt and a summary.json with the full training history and calibration.

Usage

import torch, json
from huggingface_hub import hf_hub_download
from PIL import Image

# preprocessing must match training exactly — see the code repo's preprocess.py
ckpt = hf_hub_download("husseinelsaadi/aidetect-vit-b16", "runC/checkpoints/best.pt")
blob = torch.load(ckpt, map_location="cpu", weights_only=False)

# blob["model"]  -> state dict
# blob["extra"]["config"] -> the training configuration

The complete, runnable inference path — canonical preprocessing, whole-frame tiling, temperature scaling and the tuned threshold — is the Space's app.py. Copy it rather than reimplementing: a preprocessing mismatch silently costs several points of accuracy.

Decision rule. Apply temperature scaling (summary.json → temperature), then threshold at the published operating point rather than 0.5. The threshold was fitted on held-out out-of-distribution data, because on a live demo the expensive mistake is accusing a real photograph, not missing one synthetic image.

Training

The control that makes the numbers meaningful

In the source data, fakes are PNGs straight from the generator and reals are JPEGs from photo datasets. Handed that raw, a model scores ~99% by learning "PNG means fake" and has learned nothing about AI. Every image of both classes therefore passes through one re-encoding function with no branch on the label.

The check that this worked: training the same configuration on randomly permuted labels gives 0.511 accuracy — chance. The classes are not separable by format, resolution or compression, so the reported numbers reflect generated-image structure rather than a pipeline artifact.

Limitations and intended use

• Evidence, not proof. Academic coursework, not a forensic instrument. Do not use it to make consequential accusations about a person or a publication.
• About 1 in 11 genuine photographs from unfamiliar camera pipelines is still flagged as AI. Unfamiliar sensor and compression statistics resemble generation.
• Low-resolution images are harder: CIFAKE at 32×32 scores 0.720 against 0.976 in-distribution. Heavy upscaling is its own distribution shift.
• Screenshots, memes and heavily edited photographs sit between the two classes.
• Generators released after training will drift away from what was measured here.
• Non-commercial use only — the training data is CC-BY-NC-SA / CC-BY-NC.

Citation

@misc{elsaadi2026aidetect,
  author = {El Saadi, Hussein},
  title  = {AI Image Detector: CLIP-ViT-B/16 trained across 1,100+ generators},
  year   = {2026},
  note   = {CSC625 Deep Learning, Modern University for Business and Science (MUBS)},
  url    = {https://huggingface.co/husseinelsaadi/aidetect-vit-b16}
}

References

• Bird & Lotfi (2024), CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Images, IEEE Access 12:15642–15650
• Wang et al. (2020), CNN-Generated Images Are Surprisingly Easy to Spot… For Now, CVPR
• Ojha et al. (2023), Towards Universal Fake Image Detectors that Generalize Across Generative Models, CVPR
• Park & Owens (2025), Community Forensics: Using Thousands of Generators to Train Fake Image Detectors, CVPR
• Livernoche et al. (2025), OpenFake: An Open Dataset and Platform Toward Real-World Deepfake Detection, arXiv:2509.09495

---

CSC625 Deep Learning · Hussein El Saadi · Modern University for Business and Science (MUBS) · Summer 2026.