vanguard-moderation-checkpoints

Vanguard — Moderation Checkpoints

Trained heads for multimodal-content-moderation. Each is a small classifier over a frozen CLIP ViT-B/32 backbone, which is not included here — it is loaded from openai/clip-vit-base-patch32 at runtime.

Six checkpoints: three arms on each of two benchmarks.

Each benchmark keeps its own trio. A Hateful Memes model's misinformation head never saw a misinformation label, so serving it would produce a confident number with nothing behind it.

Calibration

Each file carries a temperature fitted on validation. Logits must be divided by it before softmax. The models are severely overconfident without it — expected calibration error was 0.28 for the Hateful Memes fusion arm — and temperature scaling reduces that to 0.036 without changing a single prediction, since dividing by a positive scalar cannot move an argmax.

The size of the temperature is itself the finding: 8.60 is a large correction, and training loss reached 0.0008 while validation plateaued.

Result, stated honestly

Cross-attention beats late fusion on Hateful Memes by 0.0125 macro-F1 at p = 0.051, which is not significant at the conventional threshold. On Fakeddit the two are indistinguishable (p = 0.596). The split is consistent with Hateful Memes being constructed so neither modality alone is offensive, while Fakeddit's text frequently carries the label by itself.

Loading

import torch
from huggingface_hub import hf_hub_download

path = hf_hub_download("Aadithya1122/vanguard-moderation-checkpoints",
                       "cross_attention__hateful_memes.pt")
blob = torch.load(path, map_location="cpu", weights_only=False)
blob["state_dict"], blob["config"], blob["temperature"]

Trained on a MacBook Air M4 (MPS). No CUDA anywhere in the pipeline.