clip-vit-b32-misinfo

CLIP ViT-B/32 — Multimodal Misinformation Detection on FakeTT

Authors: Andrei-Gabriel Radu, Ciprian-Octavian Truică, Elena-Simona Apostol

National University of Science and Technology POLITEHNICA Bucharest

Supervised multimodal misinformation classification checkpoint based on CLIP ViT-B/32, trained and evaluated on FakeTT.

This model accompanies the bachelor thesis Misinformation Detection in Social Media Videos.

Results

Model

• Architecture: CLIP ViT-B/32
• Task: Binary misinformation classification
• Modality: Text + video
• Dataset: FakeTT
• Number of classes: 2
• Checkpoint: pytorch_model.bin
• Primary metric: Macro-F1

Saved Metadata

• batch_size: 8
• dropout: 0.2
• early_stopping_patience: 3
• freeze_backbone: False
• fusion_hidden_dim: 256
• gradient_accumulation_steps: 1
• learning_rate: 1e-05
• lr_scheduler_type: linear
• max_grad_norm: 1.0
• model_name: openai/clip-vit-base-patch32
• num_labels: 2
• num_train_epochs: 30
• text_max_length: 77
• warmup_ratio: 0.0
• weight_decay: 0.01

The original machine-readable metadata is included as model_metadata.json.

Usage

This repository contains a checkpoint for the custom multimodal classifier used in the thesis implementation. It is not a drop-in AutoModel.from_pretrained() repository.

Instantiate the matching custom classifier, load the included tokenizer and image/video processor, then load the state dictionary:

import torch

state = torch.load("pytorch_model.bin", map_location="cpu")

# model = MatchingClassifier(...)
# model.load_state_dict(state)
# model.eval()

The constructor and preprocessing must match the thesis implementation and model_metadata.json.

Intended Use

Research, benchmarking and reproducibility of supervised multimodal misinformation detection on short social-media videos.

Limitations

The model performs classification rather than factual verification, can inherit biases from FakeTT and its pretrained encoders, and requires the matching custom implementation for reproducible inference.

Citation

@thesis{radu2026misinformation,
    author = {Radu, Andrei-Gabriel and Truică, Ciprian-Octavian and Apostol, Elena-Simona},
    title  = {Misinformation Detection in Social Media Videos},
    school = {National University of Science and Technology POLITEHNICA Bucharest},
    year   = {2026}
}