convnext-ai-detector-finetuned

ConvNeXt V2 - Fine-Tuned AI-Generated Image Detector (Social Media Domain Adaptation)

A convnextv2_base model (via timm), fine-tuned to classify images as real vs. AI-generated, with training specifically adapted for content that has passed through social media platforms (Facebook, Instagram, TikTok).

What makes this fine-tune different

Rather than training on clean, unmodified images, this model was fine-tuned with a domain-adaptation pipeline designed to close the gap between training data and real-world social media content:

• Base dataset: a custom real-vs-AI-generated image/video-frame dataset, combined with genuinely social-media-sourced images from ComplexDataLab/OpenFake's reddit config (real photos from photography subreddits, AI-generated images from AI-generation subreddits — carrying actual platform compression rather than simulated artifacts).
• Synthetic social media augmentation: simulated UI overlays, watermarks, clickbait text, screenshot cropping, and JPEG re-compression (including double-compression, mimicking a re-share/forward).
• Real preprocessing-pipeline alignment: training images are also run through the same platform-specific preprocessing used at inference time — caption/subtitle-band cropping, infographic border stripping, Instagram-style color normalization, unsharp masking, and face-based photographic ROI extraction (via YuNet) — so the model is trained on the same kind of input it's actually scored on in production, not a mismatched approximation of it.
• Class-weighted loss to correct for imbalance introduced by the merged dataset.

Usage

import torch
import timm
from huggingface_hub import hf_hub_download

REPO_ID = "your-username/convnext-ai-detector-finetuned"  # <-- replace with your actual repo id
WEIGHTS_FILENAME = "best_convnext_v2_model.pth"

weights_path = hf_hub_download(repo_id=REPO_ID, filename=WEIGHTS_FILENAME)

model = timm.create_model("convnextv2_base", pretrained=False, num_classes=2)
state_dict = torch.load(weights_path, map_location="cpu")
model.load_state_dict(state_dict)
model.eval()

Preprocessing

Use the same resize/crop/normalize pipeline as training's validation transform:

from torchvision import transforms

_convnext_mean = (0.485, 0.456, 0.406)
_convnext_std = (0.229, 0.224, 0.225)

infer_transform = transforms.Compose([
    transforms.Resize(288, interpolation=transforms.InterpolationMode.LANCZOS),
    transforms.CenterCrop(256),
    transforms.ToTensor(),
    transforms.Normalize(_convnext_mean, _convnext_std),
])

Running inference

from PIL import Image
import torch.nn.functional as F

image = Image.open("your_image.jpg").convert("RGB")
tensor = infer_transform(image).unsqueeze(0)

with torch.no_grad():
    logits = model(tensor)
    probs = F.softmax(logits, dim=1)[0]

print(probs)  # see "Class order" below for how to read this

Class order

⚠️ Fill this in before publishing — this model was trained with torchvision.datasets.ImageFolder, which assigns class indices alphabetically by folder name. The training notebook prints this at runtime as Classes found: [...].

Replace the placeholders below with your actual training output:

Getting this backwards will silently invert every prediction downstream (this has bitten this exact model before — double check it).

Training details

• Architecture: convnextv2_base (ImageNet-pretrained, fully fine-tuned)
• Input size: 256x256 (resized from 288 via Lanczos, then center/random cropped)
• Optimizer: AdamW, learning rate 3e-5, cosine annealing schedule
• Loss: class-weighted cross-entropy
• Batch size: 16 (gradient accumulation over 2 steps, effective batch size 32)
• Epochs: 10
• Regularization: gradient clipping (max norm 1.0)
• Precision: mixed precision (AMP) with gradient scaling

Known limitations

• Performance varies by platform (Facebook / Instagram / TikTok) — evaluate against your own calibration set before deploying, especially if your target platform's content differs from the training distribution above.
• The reddit-sourced portion of training data is licensed CC-BY-NC-4.0 for its proprietary-generator subsets — check licensing before commercial use of this model.
• Face-based ROI extraction (used both in training and recommended at inference) depends on the YuNet face detector; images without detectable faces fall back to a fixed-crop heuristic, which may be less reliable.

Intended use

Binary classification of images (or video frames) as real vs. AI-generated, particularly for content sourced from or resembling social media platforms. Not intended as a sole arbiter of authenticity — use as one signal within a broader verification pipeline (e.g. alongside a dedicated deepfake/face-manipulation detector).