ijmond-smoke-segmentation

IJmond Smoke Segmentation — Pixel-wise Soft Weighting for Noisy Pseudo-labels

Master's Thesis · Vrije Universiteit Amsterdam · 2026
Author: Wang Mingjia  |  Supervisor: Yen-Chia Hsu

---

Overview

This repository contains all model checkpoints, experiment notebooks, and results from a thesis studying whether pixel-wise soft-weighting can improve smoke segmentation when supervision comes from noisy SAM-generated pseudo-labels.

Short answer: No — under multi-seed evaluation, no weighting variant reliably outperforms the plain pseudo-label baseline (M2). All six methods achieve 0.164–0.185 mIoU with overlapping standard deviations. The single-run apparent range (0.060–0.220 mIoU) was an artefact of random initialisation, not a reliable property of the loss designs.

---

Architecture

---

Methods

Six loss formulations are compared. All share the same DeepLabV3+ network at inference; differences are only in how per-pixel cross-entropy losses are aggregated during training.

Notation

• $P_{\text{label}}(i)$ — Softmax probability of the pseudo-labeled class at pixel $i$ (detached from the gradient graph)
• $P_{\text{local}}(i)$ — 5×5 average-pool of $P_{\text{label}}$ (spatial neighbourhood mean)

Method table

Key design choice: $P_{\text{label}}$ is detached from the computation graph before being used as a weight. Without detach, the model can minimise the loss by driving $P_{\text{label}} \to 0$ (zeroing the weight) instead of improving predictions.

---

Results

Primary evidence — Multi-seed evaluation (3 seeds × 6 methods)

Mean ± 1 std over 3 independent training runs (seeds 0, 1, 2). Error bars show seed standard deviation. Threshold = 0.5.

No weighting variant reliably outperforms M2. All differences lie within seed standard deviation.

Notable cross-seed patterns that are reliable:

• M3 has the lowest seed variance (σ = 0.005): binary keep/discard is more robust to initialisation than soft weighting.
• M4 has the highest mean Precision (0.481) but the lowest Recall (0.202) across all three seeds: the EMA teacher acts as a conservative regulariser that suppresses uncertain detections.

Per-seed breakdown

<details> <summary>Click to expand full per-seed mIoU table</summary>

Full metrics (mIoU, Dice, Precision, Recall) per seed: results/multiseed/per_seed_results.csv

</details>

Threshold sensitivity

Threshold sweep 0.3 → 0.7 on standalone checkpoints (single run). Method rankings under the standalone run are not reproduced in multi-seed evaluation; the sweep confirms the chosen threshold of 0.5 is not a cherry-picked operating point.

Label budget (RQ2 sub-experiment)

Each method re-trained at 25 / 50 / 75 / 100% of the 167-image training pool (single seed, fixed split).

M1-Cons tracks M2 closely across all budget levels. M1-Conf and M1-Both are most unstable at very small budgets (normalised-sum denominator can collapse when batch contains few confident pixels).

---

Deployment Benchmarks (RQ3)

All M1 variants and M2/M3 share the same deployed network — weighting only affects the training loss, not the model architecture. M4 also deploys as a single network (student only).

Inference latency

5 warm-up + 20 timed runs. CPU: Apple M-series. GPU: Tesla P100.

Training-step overhead vs M2

M4's overhead comes from a second forward pass (teacher) + EMA parameter update every step. CPU fluctuations for M3/M1-Conf are within timing noise.

---

Qualitative Analysis

Three representative prediction patterns from the 2,074-image test set.
Overlays show raw Softmax smoke probability; pixels with P < 0.05 are hidden.

Precision-contrast example — M2 over-predicts, weighting variants stay focused

M2 (purple) generates a large false-positive blob covering industrial structures. M1-Conf (red), M1-Cons (orange), and M1-Both (teal) assign substantially lower probabilities to the same non-smoke region.

---

Positive-detection example — all methods agree on genuine smoke

All four methods consistently localise the chimney plume with similar spatial extent. Weighting variants do not suppress genuine smoke signals when pseudo-labels are unambiguous.

---

Consistency-effect example — M1-Conf and M1-Cons diverge

M1-Conf (red) assigns moderate probability to a diffuse region near the chimney top. M1-Cons (orange) suppresses this prediction because the local 5×5 neighbourhood average deviates from the pixel's own confidence — illustrating how consistency weighting can reduce ambiguous false positives but may also suppress genuine detections.

40 full-resolution figures (fp / tp / cons categories) are in figures/qualitative/.

---

How to Load a Checkpoint

import torch
import segmentation_models_pytorch as smp

def load_model(ckpt_path, device="cpu"):
    model = smp.DeepLabV3Plus(
        encoder_name="mobilenet_v2",
        encoder_weights=None,
        in_channels=3,
        classes=2,
    ).to(device)
    model.load_state_dict(torch.load(ckpt_path, map_location=device))
    model.eval()
    return model

# Example: load M2 multi-seed seed-0 checkpoint
from huggingface_hub import hf_hub_download
ckpt = hf_hub_download(
    repo_id="Minggg0518/ijmond-smoke-segmentation",
    filename="checkpoints/multiseed/M2_seed0_best.pth",
)
model = load_model(ckpt)

Run inference

import numpy as np
import torch
import torch.nn.functional as F

# Load a 400×400 image (.npy format as used in IJmond dataset)
image_np = np.load("your_image.npy").astype(np.float32)          # H×W×3
t = torch.from_numpy(image_np).permute(2, 0, 1).unsqueeze(0) / 255.0
t = F.interpolate(t, size=(400, 400), mode="bilinear", align_corners=False)

with torch.no_grad():
    logits = model(t)                                              # 1×2×400×400
    prob_smoke = F.softmax(logits, dim=1)[0, 1]                   # 400×400
    pred_mask = (prob_smoke > 0.5).int()                           # binary, threshold=0.5

---

Repository Structure

├── checkpoints/
│   ├── multiseed/          # 18 checkpoints — 6 methods × 3 seeds (PRIMARY)
│   ├── standalone/         # 6 standalone reference checkpoints
│   └── pretrained_smoke5k.pth
├── experiments/
│   ├── 00_handbook.ipynb   # Unified experiment handbook (all chapters)
│   ├── 01_training.ipynb
│   ├── 02_qualitative_analysis.ipynb
│   ├── 03_effectiveness_evaluation.ipynb
│   ├── 04_label_budget_analysis.ipynb
│   └── 05_deployability_cost.ipynb
├── results/
│   ├── multiseed/          # per_seed_results.csv · summary_mean_std.csv
│   ├── standalone/         # standalone evaluation CSVs + figures
│   ├── budget/             # label budget CSVs + figures
│   └── cost/               # inference + training overhead CSVs
└── figures/
    ├── qual_precision_contrast.png
    ├── qual_positive_detection.png
    ├── qual_consistency_effect.png
    └── qualitative/        # 40 full-resolution comparison figures

---

Key Takeaways

1. Multi-seed evaluation is essential. A single training run produced apparent extremes (M1-Cons 0.220, M1-Conf 0.060) that did not reproduce across seeds. The true cross-seed range is only ~0.012 mIoU.

2. No weighting variant is reliably better than plain pseudo-labeling (M2) under the tested conditions (167-image fine-tuning set, Smoke5K pre-training, 30-epoch cosine annealing).

3. M1-Conf has a latent confirmation-bias failure mode. When the model drifts background-biased, $W = P_{\text{label}}$ gates out the corrective gradients, and the normalised-sum amplification accelerates convergence to a degenerate minimum. This occurred once (mIoU 0.060) but did not reproduce in 3-seed evaluation.

4. M4 (Mean Teacher) costs +25–30% training compute per step with no consistent accuracy gain — a poor accuracy-cost trade-off on this dataset.

5. All M1 variants are free at inference (same model, same speed). They are safe to try even when compute budget is tight.

---

Citation

Wang Mingjia (2026). Pixel-wise Soft Weighting for Smoke Segmentation under
Noisy Pseudo-labels: A Multi-seed Ablation on IJmond Industrial Data.
Master's Thesis, Vrije Universiteit Amsterdam.
Supervisor: Yen-Chia Hsu.