GOOSE-M2F

GOOSE-M2F: Adapting Mask2Former for High-Fidelity, Long-Tailed Fine-Grained Semantic Segmentation in Unstructured Outdoor Terrain

Jyothiraditya Lingam, Nikhileswara Rao Sulake, Sai Manikanta Eswar Machara

Department of Computer Science and Engineering Rajiv Gandhi University of Knowledge Technologies (RGUKT), Nuzvid, Andhra Pradesh, India

<p align="center"> <a href="https://arxiv.org/abs/2606.15937"><b>📄 Paper</b></a> • <a href="https://github.com/Aditya-Lingam-9000/GOOSE-M2F"><b>💻 Code</b></a> • <a href="https://huggingface.co/XYZ9843/GOOSE-M2F"><b>🤗 Hugging Face</b></a> • <a href="https://www.codabench.org/competitions/14257"><b>🏆 Challenge</b></a> </p>

GOOSE-M2F is a task-specific adaptation of Mask2Former for the GOOSE 2D Fine-Grained Semantic Segmentation Challenge (ICRA 2026). The proposed framework addresses long-tailed semantic segmentation in unstructured outdoor environments through enhanced object query capacity, feature refinement, auxiliary supervision, class-balanced optimization, and robust multi-scale inference.

Official Challenge Performance: 70.08% Composite mIoU (63.55% Fine mIoU, 76.61% Coarse mIoU), achieving 3rd Place on the GOOSE 2D FGSS Challenge Leaderboard.

---

📢 News

• [ICRA 2026] GOOSE-M2F achieved 3rd Place in the GOOSE 2D Fine-Grained Semantic Segmentation Challenge.
• [2026] Source code and trained models released.
• [2026] Technical report available on arXiv.

---

What is GOOSE-M2F?

The GOOSE dataset presents one of the most challenging real-world segmentation benchmarks: 64 fine-grained classes across diverse unstructured outdoor environments including forests, gravel paths, construction zones, and agricultural terrain — with a severely long-tailed class distribution.

GOOSE-M2F extends the baseline Mask2Former (Swin-Large backbone) with three key modifications engineered specifically for this challenge:

---

Architecture

Input Image [B, 3, H, W]
      │
      ▼
Swin-Large Backbone (Hierarchical, 4 stages)
  Stage 1-4: channels {192, 384, 768, 1536}, resolutions {H/4 → H/32}
      │
      ▼
MSDeformAttn Pixel Decoder (6-layer FPN)
  Output: mask_features [B, 256, H/4, W/4]
      │
      ├──────────────────────────────────────┐
      ▼                                      ▼
[NEW] Feature Refinement Module        [NEW] Auxiliary Head
  ASPP-lite: dilations {1, 3, 6, 12}     Conv(256→256→64)
  + Global Average Pooling               DB-weighted CE loss
  + CBAM Dual-Attention (Ch + Sp)        Supervised at H/4
      │
      ▼
Transformer Decoder (9 layers)
  [MOD] 200 Object Queries (was 100)
  Masked Cross-Attention
      │
      ▼
Class Head [B, 200, 65] × Mask Head [B, 200, H/4, W/4]
      │
      ▼
Hungarian Matching → Semantic Prediction

---

Training Strategy

Training Progression (V1 → V8)

---

Inference Engine

The final performance leap from 59.51% (training) to 70.08% (submission) came entirely from the inference pipeline:

---

Project Structure

goose-m2f/
├── src/
│   ├── model.py          ← GOOSEMask2Former (FRM + AuxHead + 200 queries)
│   ├── features.py       ← Dataset, augmentations, EMA, metrics
│   ├── train.py          ← Training engine (Trainer class)
│   └── inference.py      ← Dense Gaussian patch-blending inference
├── configs/
│   ├── train_config.yaml ← All training hyperparameters
│   └── infer_config.yaml ← TTA and inference settings
├── data/raw/             ← Dataset (symlink or copy)
├── models/               ← Manually placed checkpoints
├── outputs/
│   ├── checkpoints/      ← best_model.pth, latest.pth, charts
│   └── predictions/      ← Output PNG predictions
├── tests/
│   └── test_model.py     ← pytest unit tests
├── instructions/
│   └── instructions.md   ← Full setup + usage guide
└── requirements.txt

---

Quick Start

1. Setup

git clone https://github.com/Aditya-Lingam-9000/GOOSE-M2F
cd GOOSE-M2F

conda create -n goose python=3.11 -y && conda activate goose
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt
accelerate config   # Configure for your GPU setup

2. Configure Paths

Edit configs/train_config.yaml:

data_dir: "/path/to/goose_dataset"
csv_path: "/path/to/goose_label_mapping.csv"
output_dir: "outputs/checkpoints/session_01"

3. Train

# Single GPU
python -m src.train --config configs/train_config.yaml

# Multi-GPU
accelerate launch --num_processes 2 -m src.train --config configs/train_config.yaml

4. Inference

Edit configs/infer_config.yaml with the checkpoint path and image directory, then:

python -m src.inference --config configs/infer_config.yaml

5. Tests

pytest tests/ -v

---

Results

Official Leaderboard Performance (Final Submission)

Coarse Category Breakdown

---

Requirements

See requirements.txt for the complete list.

---

Citation

If you use this work, please cite:

@techreport{lingam2026goosem2f,
  title     = {GOOSE-M2F: Adapting Mask2Former for High-Fidelity, Long-Tailed Fine-Grained Semantic Segmentation in Unstructured Outdoor Terrain},
  author    = {Jyothiraditya Lingam and Nikhileswara Rao Sulake and Sai Manikanta Eswar Machara},
  year      = {2026},
  institution = {Rajiv Gandhi University of Knowledge Technologies (RGUKT)}
}

---

References

• Mask2Former: Cheng et al., Masked-Attention Mask Transformer for Universal Image Segmentation, CVPR 2022
• Swin Transformer: Liu et al., ICCV 2021
• CBAM: Woo et al., Convolutional Block Attention Module, ECCV 2018
• DeepLab: Chen et al., Rethinking Atrous Convolution, TPAMI 2017

---