age-gender-race-prediction

Perceived age, gender and race from advertising faces

Three ConvNeXt classifiers that read perceived age, gender and race from a face, together with the exact preprocessing and decision rule they were validated with. They were built for a study of demographic representation in advertising imagery, and they are benchmarked on that domain — real display advertisements and AI-generated advertisements — rather than only on the datasets they were trained on.

v2 — the race label set changed. Earlier versions of this repository shipped a 7-class race head (White / Black / Indian / East Asian / Southeast Asian / Middle Eastern / Latino_Hispanic) and a confusion-matrix count correction. Both are gone. The head is now four US-Census-style classes and the correction is a per-domain logit offset. Code written against v1 will not work unchanged.

Quick start

from huggingface_hub import snapshot_download
import sys
d = snapshot_download("TimmaJ/age-gender-race-prediction")
sys.path.insert(0, d)

from predict import DemographicPredictor
p = DemographicPredictor(domain="real")        # "ai" for generated imagery
p.predict("advert.jpg")
# {'age': 43.5, 'age_group': '35-44', 'gender': 'male', 'p_female': 0.028,
#  'race': 'Black', 'race_probs': {...}, 'domain': 'real',
#  'face_found': True, 'detector': 'insightface'}

predict.py runs the whole pipeline: InsightFace detection, a YOLOv11n-face fallback, a 0.35 margin expansion, the square resize, and the calibrated race decision. Use it rather than calling the checkpoints directly — the two steps below are easy to get wrong and both cost real accuracy.

domain is not a convenience flag

The classifier behaves differently on photographs and on generated images, so the race offsets were fitted separately on each. Pass "real" for photographs and "ai" for anything a generator made. Passing the wrong one moves the race distribution by several percentage points.

The crop margin matters more than it looks

Predicting on the whole image instead of a padded face crop costs 7.2 points of accuracy and 16.5 points of macro-F1 (78.0 → 70.8, 66.8 → 50.3, same 1,000 human-labelled ads). benchmark/crop_vs_full_image.csv has the per-image predictions.

How the race label is decided

Not a plain argmax on the raw logits. In order:

1. Temperature-scale the logits by T = 0.909, fitted on the 1,000 human-labelled real ads.
2. Add per-domain class offsets (calibration.py, BIAS_REAL / BIAS_AI).
3. Take the argmax.

The offsets are per-domain maximum-likelihood calibration on 2,000 human-labelled faces, plus one additional +1.0 offset on Other — the rare class was being predicted at 26% recall while the other three sat at 75–94%. Both steps were selected on label-based metrics (macro-F1, Other F1) and never on closeness to any census target.

Temperature alone is monotonic and cannot change an argmax, but it scales the logits against the fixed offsets, so it is part of the rule and must not be dropped.

Benchmarks

Two independent sets of 1,000 faces, each labelled by three human raters for perceived age, gender and race, sampled from the domains the models are meant to be used on. Full tables in benchmark/.

Race — where these models are actually better than the alternatives:

Gender and age — competitive, not dominant, and worth saying plainly:

CLIP matches it on gender and MiVOLO matches it on age error. The reason to use this one is the race head and the fact that every number above was measured on advertising faces rather than on a held-out slice of the training set.

Human raters agreed with each other at Krippendorff's α = 0.80 (real) and 0.84 (AI) on race, 0.95 on gender, and 0.96 on age.

Limitations — read this before reporting a race distribution

Other is weak, and it is weak in a way that biases results. Per-class, on real ads: precision 37%, recall 53%, F1 44. On AI ads: precision 35%, recall 63%, F1 45. Other here covers Hispanic/Latino, Native American and multiracial appearance — the classes a four-way forced choice serves worst.

A separate validation on 449 generated faces (299 where a Latino person had been requested, 150 White-requested controls) found human raters identified a Latino face in 58.5% of them where this rule finds 39.1%, against 98.7% agreement on the controls. Reported Other shares are systematically conservative. If your conclusion depends on the size of the Other group, treat the model's number as a floor and validate against human labels.

Other things worth knowing:

• These are perceived attributes, annotated from appearance by human raters and then learned. They are not anyone's identity, self-description, or ancestry. Do not present them as such.
• Race is a forced choice among four classes. Middle Eastern and North African faces fall in White, South Asian and Pacific Islander in Asian, following US Census convention. There is no "uncertain" output.
• Hispanic origin is not a face class. In the census it is a separate question, and it is folded into Other here.
• One face per image. predict.py keeps the highest-scoring detection.
• Trained on FairFace and UTKFace, validated on advertising. Performance on other domains — CCTV, medical imaging, historical photographs — is unmeasured.
• Age is a regression clipped to 0–120. Band accuracy near the boundaries is limited by the fact that FairFace ages were imputed within bands during training.

Do not use these models to make decisions about individuals — hiring, lending, access, identification, law enforcement. They are measurement instruments for population-level analysis of image corpora, and the Other numbers above should make clear why an individual prediction is not trustworthy.

Files

age/ gender/ race/     config.json, model.safetensors, preprocessor_config.json,
                       train_args.json, val_metrics.json
predict.py             the full pipeline: detect, crop, resize, calibrate, predict
calibration.py         the race decision rule; _bias_real.npy / _bias_ai.npy
example.py             quickstart
requirements.txt
benchmark/             head-to-head tables, per-class race metrics, confusion
                       matrices for both domains, rater agreement, the crop
                       ablation, and calibration.json

The preprocessor_config.json files are ViTImageProcessor configs on purpose: the models were trained on a plain square resize with no aspect-preserving resize and no centre crop, which the ConvNeXt processor cannot express. They reproduce the training transform exactly.

Licence

MIT for the code. Weights are for non-commercial research use. Human labels are CC BY 4.0. Training data comes from FairFace and UTKFace under their own licences.

Citation

@article{jansen2026representation,
  title   = {Census-grounded prompt augmentation reduces demographic misrepresentation in AI-generated advertising},
  author  = {Jansen, Tijmen},
  year    = {2026},
  note    = {Manuscript in preparation}
}