community-forensics-frontier-detector-2026-08

Community Forensics frontier detector (August 2026)

This is an independently fine-tuned whole-image AI-generated-image detector based on the MIT-licensed OwensLab Community Forensics 384 model. It is not an official OwensLab release.

The release is intentionally accompanied by complete positive and negative validation results. It performs strongly on ordinary, web-laundered, and moderately degraded images, but it is not robust to every declared attack. In particular, very small synthetic composites and very-low-resolution inputs remain serious weaknesses.

Files

• model.safetensors: canonical EMA inference weights.
• community_forensics_frontier_fp16.onnx: recommended compact deployment artifact.
• config.json: architecture, preprocessing, labels, and decision threshold.
• calibrator.json: frozen calibration parameters.
• reports/final_report.json and .md: complete final evaluation.
• reports/redteam_report.json and .md: complete 33-condition robustness evaluation.
• metadata/training.json and metadata/data_audit.json: training and dataset-audit records.
• metadata/preparation.json: construction record for the combined manifest.
• metadata/dataset_composition.json: row counts by role, label, source, license, repository, and revision.
• metadata/canary_summary.json: equal-exposure architecture comparison used before the final run.
• metadata/ranking.json: frozen candidate selection and calibrator.
• metadata/training_recipe.json: machine-readable optimizer, augmentation, sampling, and validation recipe.
• configs/redteam.yaml: exact 33-condition robustness registry.

Intended use

Binary whole-image classification:

• 0: real
• 1: AI-generated

This model is a screening signal, not proof of provenance. Do not use it as the sole basis for accusations, moderation sanctions, employment decisions, or legal conclusions. It was not trained as a localization model and should not be expected to reliably identify an image containing only a tiny synthetic patch.

Preprocessing and decision rule

• RGB input
• resize the short edge to 440 pixels
• center-crop to 384 × 384
• normalize using the values in config.json
• output: one raw logit
• frozen PyTorch-reference decision rule: raw_logit >= 1.359375 means AI-generated
• equivalent calibrated probability threshold: 0.65

The ONNX model uses FP16 weights with FP32 input and output. The PyTorch calibration ledger was produced with CUDA bfloat16 autocast. This numerical-runtime difference is reported below rather than hidden.

Dataset construction

The final manifest combined an older 73,371-row forensic corpus with 40,101 newly selected frontier-generator images. The public frontier collection is available at Thermostatic/frontier-synthetic-images-2026; the public dataset is the new frontier component, not a redistribution of every image in the combined training manifest.

The five real-image training sources were:

The 68,247 synthetic training rows came from 131 source buckets. They include older generators and forensic benchmarks—such as SD 1.5, GLIDE, Wukong, Midjourney, SynthBuster, and OpenFake—plus 40,101 selected frontier images from 42 named generator buckets. Those frontier buckets include GPT Image generations, DALL·E 3, FLUX 1/2 variants, Imagen 3/4, Seedream 3/4/5, Qwen Image variants, Hunyuan Image, Midjourney 5.2, Recraft, Ideogram, Janus, Sana, Z-Image, and others. Exact per-source counts and source revisions are in metadata/dataset_composition.json and metadata/preparation.json.

The calibration split contains 1,983 LAION/Pexels real images and 1,929 synthetic images across 59 OpenFake generator sources. It was kept group-disjoint from training. A further 189 recent Hugging Face images were held out for positive-only diagnostics, and the 90-image OpenRouter cohort was protected from training. In total, 4,802 images from protected manifests were excluded from frontier-data selection.

The launch audit reported zero issues. It checked the finalized manifest with an exact-banded perceptual-hash search at Hamming distance 4, alongside the stored content hashes and group assignments. The combined manifest SHA-256 is af629187fca31ab2f4f7f4c81ea3a761c13ea7cb8d4ee8a9bb9e7bd8e1eb6f36.

Dataset licenses and terms remain source-specific. The full mixed corpus is not claimed to be reproducible from the public frontier dataset alone; the release publishes composition and provenance metadata so this limitation is explicit.

Architecture selection

Four candidates received equal canary exposure: 600 steps, batch size 48, and 28,800 presented samples each. Selection used the calibration role only; the competition test remained unopened.

Community Forensics was both the strongest candidate and the healthiest eligible loss curve. A large-teacher/distillation experiment was considered but cancelled before any teacher outputs were used. This release contains no distillation. No bounty-participant detector weights, pseudo-labels, or outputs were used for training, selection, or calibration.

For clarity, the 0.9214 initial value in the Community Forensics canary row is the score of an earlier legacy-data fine-tune used to initialize that canary. It is not the untouched upstream OwensLab score. A direct comparison against the original upstream weights follows below.

Direct comparison with the original upstream model

The untouched OwensLab/commfor-model-384 revision and this release were scored with the same loader, bfloat16 inference path, images, deterministic views, and calibration procedure. Each model received its own bias-only calibrator fitted under the same protocol. The original raw-logit boundary was -9.4375; the fine-tuned boundary is 1.359375.

The protected positive-only recent-generator cohorts show the same pattern:

This is strong evidence that fine-tuning improved the original model, especially under laundering. It is not an unbiased competition-test estimate: the 3,912-image calibration set was used for development and checkpoint selection. The two recent-generator cohorts were protected from training, but they are positive-only and relatively small. The exact comparison is published in metadata/original_comparison.json.

Final training recipe

The final run initialized from the winning Community Forensics canary checkpoint, which itself initialized from OwensLab/commfor-model-384@6076002bf0d9dd37537f965ee2f06f826c333b61.

Augmentation was class-symmetric—real and synthetic images received the same transform distribution. It used:

• up to 8% crop on each edge with probability 0.18;
• scale 0.40–1.30 with probability 0.72, using nearest/bilinear/bicubic/Lanczos;
• Gaussian blur radius 0.15–0.75 with probability 0.08;
• a first JPEG/WebP round trip with probability 0.82 and an independent second round trip with probability 0.30;
• codec quality sampled uniformly from 30–95, with WebP selected 22% of the time;
• random 384 crop after resizing the short edge to 440;
• random horizontal flip with probability 0.5;
• ImageNet normalization: mean [0.485, 0.456, 0.406], standard deviation [0.229, 0.224, 0.225].

The best full-run checkpoint occurred at step 4,500. The run completed all 6,000 steps so the later validation trajectory could be checked rather than stopping immediately at the first peak.

Training and data audit summary

The final run took 14,187.6 seconds (3 h 56 min) and reported zero decode failures.

Evaluation and calibration protocol

Candidate selection, checkpoint selection, and calibration used development/calibration data only. There was no competition-test manifest in the combined data, and the locked test was never opened.

The three recurring validation views were deterministic per sample:

• clean: decode, RGB conversion, resize/crop, and normalization only;
• web: limit the long edge to 768 pixels using bilinear resampling, then JPEG quality 60 with 4:2:0 subsampling;
• hard: limit the long edge to 512 pixels using bilinear resampling, then JPEG quality 40 with 4:2:0 subsampling.

At each 750-step checkpoint, the model scored 3,912 images × 3 views = 11,736 predictions. Selection used macro balanced accuracy across views, not clean accuracy alone. Step 4,500 was selected at 0.9328 macro balanced accuracy; step 6,000 finished lower at 0.9298, so the earlier checkpoint was retained.

Calibration compared supported calibrator families out-of-fold and selected a bias-only transform. The frozen parameters are slope 1.0, intercept -0.7403357915937764, target probability threshold 0.65, and equivalent raw-logit boundary 1.359375. The candidate-ranking ledger and its SHA-256 are published in metadata/ranking.json.

The additional evaluation layers were:

1. recent-generator positive-only diagnostics on OpenRouter-90 and recent-HF-189;
2. a 33-condition red-team registry over all 3,912 calibration images (129,096 predictions);
3. 2,000-replicate hierarchical bootstrap intervals over per-image worst variants;
4. FP16 and FP32 ONNX export smoke tests;
5. full 11,736-prediction PyTorch/ONNX ledger comparison.

The red-team suite is development evidence, not an independent test. Its exact operations and severities are published in configs/redteam.yaml.

Calibration results

All values below are development/calibration results. The locked competition test was never opened.

Recent-generator diagnostics

These are small positive-only diagnostic cohorts, so recall is reported rather than balanced accuracy.

The OpenRouter result is a material limitation: detection degrades substantially after the hard laundering pipeline.

Robustness results — failed gates

The full development red-team evaluation covered 3,912 images, 33 conditions, and 129,096 predictions with zero inference failures. The report is nevertheless marked valid: false because three declared gates failed.

composite_05 places only 5% synthetic foreground into an otherwise real image. It exposes the mismatch between whole-image classification and localized synthetic-content detection; it is not omitted or reclassified here. The per-image worst-of-all-attacks balanced accuracy was 0.2766 (95% hierarchical-bootstrap interval 0.2523–0.3121). Very-low-resolution cases at 96–128 pixels were also weak. See reports/redteam_report.md for the full case table.

ONNX parity

Recommended artifact: community_forensics_frontier_fp16.onnx

An FP32 ONNX export was also evaluated. It was twice as large and produced 35 threshold disagreements, so it is not published as the recommended artifact. ONNX balanced accuracy at the frozen boundary was 0.9571 clean, 0.9373 web, and 0.9047 hard—within 0.0013 of the PyTorch-reference values.

Limitations

• Weak on severe low-resolution laundering.
• Weak on small or diluted synthetic regions because this is a whole-image classifier.
• OpenRouter hard-view recall was only 0.4778 on a 90-image cohort.
• Three important external robustness families were not covered.
• Validation results are development evidence, not an untouched competition-test estimate.
• Detectors can age quickly as image generators and laundering pipelines change.

Lineage and license

Base model revision: OwensLab/commfor-model-384@6076002bf0d9dd37537f965ee2f06f826c333b61.

• Upstream code: JeongsooP/Community-Forensics
• Upstream paper: Community Forensics
• Base model license: MIT

This fine-tuned release is published under the MIT license. Dataset licenses and terms remain those of their respective sources.