LoRA-deepfake-detection

Deepfake Detection via LoRA Fine-Tuned ViT

Binary classifier distinguishing real portrait photos from AI-generated faces. Fine-tunes a pre-trained ViT-B/16 using LoRA adapters (PEFT), keeping 99%+ of the backbone frozen while adapting only the attention projections. LoRA adapters are merged before export — no PEFT dependency at inference time.

Primary dataset: 140K Real and Fake Faces — 140 000 images, perfectly balanced, predefined train/valid/test split. Real faces from Flickr, fake faces generated with StyleGAN2.

Model Details

Usage

import numpy as np
import onnxruntime as ort
from PIL import Image
from torchvision.transforms import CenterCrop, Compose, Normalize, Resize, ToTensor

transform = Compose([
    Resize(256),
    CenterCrop(224),
    ToTensor(),
    Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

session = ort.InferenceSession("model_quint8.onnx")

img = Image.open("face.jpg").convert("RGB")
x = transform(img).unsqueeze(0).numpy()
logit = session.run(None, {"input": x})[0][0, 0]
prob_fake = float(1 / (1 + np.exp(-logit)))
print(f"Fake probability: {prob_fake:.3f}")

Results

Dataset: 140K Real and Fake Faces — 100k train / 20k val / 20k test, perfectly balanced. Model: ViT-B/16 + LoRA (r=16, target: query + value projections) Training: 10 epochs, AdamW, cosine LR with warmup, batch size 128

Classification (test set):

Quantization benchmark (CPU, 100 inference runs, batch size 1):

The model converges rapidly — 96.8% accuracy is already reached after epoch 2, with diminishing gains thereafter. LoRA keeps 99%+ of backbone parameters frozen throughout, training only ~0.68% of total parameters (590k adapter params on top of 86M ViT-B/16 backbone).

Dynamic INT8 quantization reduces model size by 4× and latency by 3× with a negligible 0.16 percentage point accuracy drop.