2026-24679-water-image-automl-classifier

Water Presence AutoML Classifier

Model Summary

This repository contains the best image-classification model selected through a fixed-budget neural-network AutoML search.

The model predicts whether a scene contains a visibly identifiable body of water.

• Label 0: no_water
• Label 1: water
• Framework: AutoGluon MultiModal
• Source dataset: ssg1/places-water-binary
• Selected trial: mobilenetv3_adamw_lr1e4
• Selected backbone: mobilenetv3_small_100
• Input format: 224 x 224 RGB image

Intended Use

This model was created for CMU 24-679 Homework 2 to demonstrate neural-network AutoML.

Appropriate uses include:

• Coursework demonstrations
• Small-scale image-classification experiments
• Comparing compact pretrained architectures
• Testing AutoGluon model persistence

The model is not intended for:

• Flood or drowning detection
• Emergency response
• Maritime navigation
• Environmental monitoring
• Surveillance
• Safety-critical decisions

Dataset

The source dataset contains 34 independent original photographs. Parent-aware splitting prevents an original image and its augmented children from crossing splits.

The complete set of original images is balanced:

• 17 no_water
• 17 water

Validation and test contain only independent, unaugmented original photographs.

Preprocessing and Augmentation

The source images were converted to RGB, square-cropped, and resized to 224 x 224 pixels.

The training split includes:

• Horizontal flipping
• Small rotation
• Brightness and contrast changes
• Colour jitter

Augmentation was applied only to training parents. Validation and test contain original images only.

The model uses AutoGluon's resize_to_square transform. Backbone-specific normalization is handled by AutoGluon and TIMM.

Leakage Prevention

The notebook verifies that parent_id values are disjoint across training, validation, and test.

This prevents an original photograph from appearing in one split while an augmented version appears in another.

AutoML Search

The search compares:

• ResNet-18
• MobileNetV3-Small
• EfficientNet-B0

The searched hyperparameters include:

• Backbone architecture
• Learning rate
• Optimizer
• Weight decay

Every trial receives the same:

• Maximum time: 300 seconds
• Maximum epochs: 20
• Early-stopping patience: 6 validation checks
• Per-GPU batch size: 8
• Random seed: 24679

The maximum planned search budget is 1800 seconds.

The measured total trial time was 218.1 seconds.

• Completed trials: 6
• Failed trials: 0

Search Results

Validation accuracy is the primary selection metric. Validation balanced accuracy is the first tie-breaker, followed by shorter fitting time.

Selected Architecture and Hyperparameters

{
  "trial_id": "mobilenetv3_adamw_lr1e4",
  "backbone": "mobilenetv3_small_100",
  "learning_rate": 0.0001,
  "optimizer": "adamw",
  "weight_decay": 0.0001
}

Best validation accuracy:

1.0000

Best validation balanced accuracy:

1.0000

Early Stopping and Training Curves

Training used a maximum of 20 epochs, early-stopping patience of 6 validation checks, and a per-trial limit of 300 seconds.

The repository includes best_model_training_curves.png, which was extracted from the selected trial's TensorBoard scalar logs.

Held-Out Test Results

The test split was used only after model selection.

Baseline Comparison

The baseline always predicts the majority class among the independent training originals.

Confusion Matrix

Test Predictions

Statistical Uncertainty

The test set contains only 6 independent images.

• Correct predictions: 4/6
• Accuracy: 0.6667
• Wilson 95% confidence interval: [0.3000, 0.9032]
• One prediction changes accuracy by 16.7 percentage points

The reported metrics should be treated as preliminary coursework results because the test set is very small.

Error Analysis

The selected model misclassified 2 of the 6 held-out images. Potential causes include small or distant water regions, reflections, blue sky, fountains, unusual lighting, and viewpoints that are not well represented in the training originals.

Known Limitations and Failure Modes

• Only 34 independent original scenes are available.
• Augmented images do not add independent scenes.
• Validation contains only five original images.
• Test contains only six original images.
• Images were captured by one photographer and phone.
• Daylight scenes are more common than night scenes.
• Water may occupy only a small part of an image.
• Reflections and blue skies may cause false positives.
• Dark or distant water may cause false negatives.
• Water type is not separately classified.

Ethical Considerations

The model should not be used for safety-critical or high-impact decisions.

Some photographs may contain distant people in public places, but people are not the subject of the task. The model was not designed to identify people, locations, or private information.

Reproducibility

• Random seed: 24679
• AutoGluon MultiModal: 1.6.1
• Datasets: 5.0.1
• Hardware: Tesla T4
• Maximum epochs: 20
• Early-stopping patience: 6
• Trial time limit: 300 seconds
• Per-GPU batch size: 8

The search space is stored in search_space.json. The selected configuration is stored in best_hyperparameters.json.

Loading the Model

import zipfile
import pandas as pd

from autogluon.multimodal import MultiModalPredictor
from huggingface_hub import hf_hub_download

repo_id = "cannj/2026-24679-water-image-automl-classifier"

archive_path = hf_hub_download(
    repo_id=repo_id,
    filename="water_image_automm_predictor.zip",
)

with zipfile.ZipFile(
    archive_path,
    "r",
) as archive:
    archive.extractall(
        "water_image_predictor"
    )

predictor = MultiModalPredictor.load(
    "water_image_predictor"
)

example = pd.DataFrame(
    dict(
        image=[
            "path/to/image.jpg"
        ]
    )
)

prediction = predictor.predict(
    example
)

print(prediction)

Repository Files

AI Usage Disclosure

Generative AI tools were used to assist with drafting, organizing, and debugging portions of the notebook code and documentation.

License

The source dataset is released under the MIT License. This coursework repository is also published under the MIT License. Users should additionally review the terms associated with the selected pretrained TIMM backbone.