multisense_resnet50_best

MultiSense Disaster & Conflict Damage Classifier

Model Description

This model is a fine-tuned ResNet50 (ImageNet pretrained) designed for general damage-severity classification, spanning both natural disasters and conflict zonesspan_14span_14. It serves as the visual-processing branch of a multimodal disaster response API pipelinespan_15span_15. The model architecture utilizes a GELU activation function in the new classifier head and was trained using AdamW optimizer, OneCycleLR, and mixed precisionspan_16span_16.

Dataset

The model was trained on the MultiSense / GAZADeepDav dataset (Mendeley Data, DOI 10.17632/krkft96n43.2)span_17span_17.

• Classes: 5 categories mapping to real events: Flooding (Derna, Libya), Earthquake (Syria), Conflict (Gaza), Hurricane (Harvey), and No Damagespan_18span_18.
• Preprocessing: Images feature a ~12% border-crop on each edge to mitigate corner and edge artifactsspan_19span_19.
• Augmentations: Training utilized RandomResizedCrop, horizontal flip, small-angle rotation, and color jitter (to counter brightness confounds)span_20span_20. A WeightedRandomSampler was used to handle a 2.3x class imbalancespan_21span_21.

Performance Metrics

The model achieved the following on the test set:

• Weighted Accuracy: 97%span_22span_22.
• Macro Average F1: 0.96span_23span_23.
• Per-class F1: hurricaine_harvey (1.00), gaza_war (0.98), derna_flood (0.96), syria_earthquake (0.96), no_damage (0.92)span_24span_24.

Known Limitations and Biases

• Visual Artifacts & Shortcut Learning: A rigorous visual audit revealed source artifacts in several classesspan_25span_25. no_damage contains photo-company watermarks and DJI logos; gaza_war includes news-broadcast text and network logos; syria_earthquake includes YouTube UI chrome; and derna_flood contains news watermarks and explicit captionsspan_26span_26. hurricane_harvey is visually clean (0/6 samples had artifacts in the audit), which creates a real risk that the model learns "no watermark = hurricane_harvey" as a shortcutspan_27span_27.
• Aspect Ratio Bias: The dataset has a bimodal aspect ratio (square vs widescreen)span_28span_28. hurricaine_harvey is a statistical outlier (81.3% square)span_29span_29. A post-training shortcut check showed a modest gap in test accuracy based on aspect ratio: square (0.960), widescreen (0.998), other (1.000)span_30span_30. This is likely because widescreen frames contain more visual context than small square tiles, but users should be aware of this confoundspan_31span_31.