Downloads · 30 days
98
23% of all-time downloads
EPFL-ECEO/segformer-b5-finetuned-coralscapesv2-1024-1024-95_class
segformer-b5-finetuned-coralscapesv2-1024-1024-95_class is a image segmentation model from EPFL-ECEO. Use it for the image segmentation task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
SegFormer model with a MiT-B5 backbone fine-tuned on CoralscapesV2 at resolution 1024x1024, on the fine-grained 95-class label set. Introduced in CoralscapesV2: Panoptic and Fine-Grained Visual Scene Understanding in…
Downloads · 30 days
98
23% of all-time downloads
All-time downloads
425
Public
Parameters
84.7M
339 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors339 MB · 100%
From the Hugging Face model README
SegFormer model with a MiT-B5 backbone fine-tuned on CoralscapesV2 at resolution 1024x1024, on the fine-grained 95-class label set. Introduced in CoralscapesV2: Panoptic and Fine-Grained Visual Scene Understanding in Coral Reefs.
nvidia/mit-b5The simplest way to segment an image is a single forward pass:
from transformers import SegformerImageProcessor, SegformerForSemanticSegmentation
from datasets import load_dataset
# Load an image from CoralscapesV2 (or use your own)
image = load_dataset("josauder/coralscapesV2", split="test")[0]["image"]
processor = SegformerImageProcessor.from_pretrained("EPFL-ECEO/segformer-b5-finetuned-coralscapesv2-1024-1024-95_class")
model = SegformerForSemanticSegmentation.from_pretrained("EPFL-ECEO/segformer-b5-finetuned-coralscapesv2-1024-1024-95_class")
inputs = processor(image, return_tensors="pt")
outputs = model(**inputs)
labels = processor.post_process_semantic_segmentation(
outputs, target_sizes=[(image.size[1], image.size[0])]
)[0].numpy() # HxW array of class IDs
For images that are far from the 1024x1024 training resolution (e.g. the native
1024x2048 CoralscapesV2 frames), a strided sliding-window gives better results.
This reproduces the strided 2 x 1024x1024 evaluation protocol reported below:
import torch
import torch.nn.functional as F
import numpy as np
from transformers import SegformerImageProcessor, SegformerForSemanticSegmentation
from datasets import load_dataset
device = "cuda" if torch.cuda.is_available() else "cpu"
def resize_image(image, target_size=1024):
"""Resize so that the shorter side equals `target_size`."""
h_img, w_img = image.size
if h_img < w_img:
new_h, new_w = target_size, int(w_img * (target_size / h_img))
else:
new_h, new_w = int(h_img * (target_size / w_img)), target_size
return image.resize((new_h, new_w))
def segment_image(image, processor, model, crop_size=(1024, 1024), num_classes=96):
"""Overlapping sliding-window inference with 1024x1024 windows."""
h_crop, w_crop = crop_size
img = torch.Tensor(np.array(resize_image(image, 1024)).transpose(2, 0, 1)).unsqueeze(0)
batch_size, _, h_img, w_img = img.size()
h_grids = int(np.round(3 / 2 * h_img / h_crop)) if h_img > h_crop else 1
w_grids = int(np.round(3 / 2 * w_img / w_crop)) if w_img > w_crop else 1
h_stride = int((h_img - h_crop + h_grids - 1) / (h_grids - 1)) if h_grids > 1 else h_crop
w_stride = int((w_img - w_crop + w_grids - 1) / (w_grids - 1)) if w_grids > 1 else w_crop
preds = img.new_zeros((batch_size, num_classes, h_img, w_img))
count_mat = img.new_zeros((batch_size, 1, h_img, w_img))
for h_idx in range(h_grids):
for w_idx in range(w_grids):
y1 = min(h_idx * h_stride + h_crop, h_img) - h_crop
x1 = min(w_idx * w_stride + w_crop, w_img) - w_crop
y2, x2 = y1 + h_crop, x1 + w_crop
crop_img = img[:, :, y1:y2, x1:x2]
inputs = processor(crop_img, return_tensors="pt")
inputs["pixel_values"] = inputs["pixel_values"].to(device)
with torch.no_grad():
logits = model(**inputs).logits
logits = F.interpolate(logits, size=crop_img.shape[-2:], mode="bilinear", align_corners=False)
preds += F.pad(logits, (x1, preds.shape[3] - x2, y1, preds.shape[2] - y2)).cpu()
count_mat[:, :, y1:y2, x1:x2] += 1
preds = preds / count_mat
preds = preds.argmax(dim=1)
preds = F.interpolate(preds.unsqueeze(0).float(), size=image.size[::-1], mode="nearest")
return preds.squeeze().cpu().numpy().astype(np.uint8)
processor = SegformerImageProcessor.from_pretrained("EPFL-ECEO/segformer-b5-finetuned-coralscapesv2-1024-1024-95_class")
model = SegformerForSemanticSegmentation.from_pretrained("EPFL-ECEO/segformer-b5-finetuned-coralscapesv2-1024-1024-95_class").to(device)
image = load_dataset("josauder/coralscapesV2", split="test")[0]["image"]
labels = segment_image(image, processor, model)
The model is trained on the CoralscapesV2 dataset, a general-purpose dense semantic segmentation dataset for coral reefs (2433 densely annotated 1024x2048 video frames from 55 reef sites in the Red Sea region).
Following the best configuration of the original Coralscapes SegFormer baseline: AdamW optimizer with an initial learning rate of 6e-5, weight decay 1e-2, and a polynomial (power 1) learning-rate schedule for a fixed 100 epochs with batch size 4 on the combined train+val split. Augmentations: random resized crop to 1024x1024 (scale 0.02-1.0), color jitter (brightness/contrast/saturation in [0.8, 1.2], hue in [-0.05, 0.05]), random rotation up to +/-15 degrees, and random horizontal flips. Inputs are normalized with the ImageNet mean and standard deviation.
Results on the CoralscapesV2 and Coralscapes (V1) test splits. This model is trained natively on the 95-class
set; we report the native 95-class metrics as well as metrics obtained by mapping the 95-class predictions to the
39-class legacy set for comparison. single-pass predicts once at 768x1376; strided runs a 1024x1024 window at a
1024px stride (2 windows per frame). Metrics are pixel accuracy and mean IoU (%).
| Prediction | V2 Acc (95) | V2 mIoU (95) | V2 Acc (39, mapped) | V2 mIoU (39, mapped) | V1 Acc (39, mapped) | V1 mIoU (39, mapped) |
|---|---|---|---|---|---|---|
| single-pass (768x1376) | 79.462 | 36.420 | 81.937 | 56.831 | 82.663 | 56.387 |
| strided (2 x 1024x1024) | 80.382 | 37.177 | 82.836 | 59.156 | 83.553 | 58.615 |
The model outputs 96 logits (IDs 0-95). ID 0 is void / unlabeled and is ignored during training and evaluation; IDs 1-95 are the CoralscapesV2 semantic classes. fish is class 25.
The colors below are the official CoralscapesV2 visualization palette. The full id2label and label2color
dictionaries are also distributed with the dataset as id2label.json and label2color.json.
| ID | Class | RGB |
|---|---|---|
| 0 | unlabeled / void (ignored) | — |
| 1 | acanthaster planci | <img src="https://placehold.co/16x16/b3f5ea/b3f5ea.png" width="14" height="14"> [179, 245, 234] |
| 2 | acropora alive | <img src="https://placehold.co/16x16/ec80ff/ec80ff.png" width="14" height="14"> [236, 128, 255] |
| 3 | acropora bleached | <img src="https://placehold.co/16x16/ffecfe/ffecfe.png" width="14" height="14"> [255, 236, 254] |
| 4 | acropora dead | <img src="https://placehold.co/16x16/763f80/763f80.png" width="14" height="14"> [118, 63, 128] |
| 5 | algae covered brain coral | <img src="https://placehold.co/16x16/d0cf3a/d0cf3a.png" width="14" height="14"> [208, 207, 58] |
| 6 | algae covered branching coral | <img src="https://placehold.co/16x16/0647f5/0647f5.png" width="14" height="14"> [6, 71, 245] |
| 7 | algae covered lobophylliidae | <img src="https://placehold.co/16x16/32d082/32d082.png" width="14" height="14"> [50, 208, 130] |
| 8 | algae covered massive coral | <img src="https://placehold.co/16x16/43b6d0/43b6d0.png" width="14" height="14"> [67, 182, 208] |
| 9 | algae covered pocillopora | <img src="https://placehold.co/16x16/e16d4d/e16d4d.png" width="14" height="14"> [225, 109, 77] |
| 10 | algae covered porites | <img src="https://placehold.co/16x16/ad5182/ad5182.png" width="14" height="14"> [173, 81, 130] |
| 11 | algae covered substrate | <img src="https://placehold.co/16x16/7da37d/7da37d.png" width="14" height="14"> [125, 163, 125] |
| 12 | algae covered table acropora | <img src="https://placehold.co/16x16/30a2dd/30a2dd.png" width="14" height="14"> [48, 162, 221] |
| 13 | anemone | <img src="https://placehold.co/16x16/00ffbd/00ffbd.png" width="14" height="14"> [0, 255, 189] |
| 14 | background | <img src="https://placehold.co/16x16/1da2d8/1da2d8.png" width="14" height="14"> [29, 162, 216] |
| 15 | brain coral alive | <img src="https://placehold.co/16x16/724e78/724e78.png" width="14" height="14"> [114, 78, 120] |
| 16 | brain coral bleached | <img src="https://placehold.co/16x16/db43ee/db43ee.png" width="14" height="14"> [219, 67, 238] |
| 17 | brain coral dead | <img src="https://placehold.co/16x16/4eac63/4eac63.png" width="14" height="14"> [78, 172, 99] |
| 18 | branching coral alive | <img src="https://placehold.co/16x16/e25b9d/e25b9d.png" width="14" height="14"> [226, 91, 157] |
| 19 | branching coral bleached | <img src="https://placehold.co/16x16/fce7f0/fce7f0.png" width="14" height="14"> [252, 231, 240] |
| 20 | branching coral dead | <img src="https://placehold.co/16x16/7b3256/7b3256.png" width="14" height="14"> [123, 50, 86] |
| 21 | cirrhipathes | <img src="https://placehold.co/16x16/06bd16/06bd16.png" width="14" height="14"> [6, 189, 22] |
| 22 | dark | <img src="https://placehold.co/16x16/1f1f1f/1f1f1f.png" width="14" height="14"> [31, 31, 31] |
| 23 | dead giant clam | <img src="https://placehold.co/16x16/599b86/599b86.png" width="14" height="14"> [89, 155, 134] |
| 24 | feather worm | <img src="https://placehold.co/16x16/9fba26/9fba26.png" width="14" height="14"> [159, 186, 38] |
| 25 | fish | <img src="https://placehold.co/16x16/ffff00/ffff00.png" width="14" height="14"> [255, 255, 0] |
| 26 | fungiidae alive | <img src="https://placehold.co/16x16/6485f8/6485f8.png" width="14" height="14"> [100, 133, 248] |
| 27 | fungiidae bleached | <img src="https://placehold.co/16x16/c88a1b/c88a1b.png" width="14" height="14"> [200, 138, 27] |
| 28 | fungiidae dead | <img src="https://placehold.co/16x16/035c2c/035c2c.png" width="14" height="14"> [3, 92, 44] |
| 29 | galaxea alive | <img src="https://placehold.co/16x16/60bc6c/60bc6c.png" width="14" height="14"> [96, 188, 108] |
| 30 | galaxea bleached | <img src="https://placehold.co/16x16/43f33c/43f33c.png" width="14" height="14"> [67, 243, 60] |
| 31 | galaxea dead | <img src="https://placehold.co/16x16/b0e10b/b0e10b.png" width="14" height="14"> [176, 225, 11] |
| 32 | giant clam | <img src="https://placehold.co/16x16/bdffea/bdffea.png" width="14" height="14"> [189, 255, 234] |
| 33 | goniopora alive | <img src="https://placehold.co/16x16/b50578/b50578.png" width="14" height="14"> [181, 5, 120] |
| 34 | goniopora dead | <img src="https://placehold.co/16x16/b3bf1e/b3bf1e.png" width="14" height="14"> [179, 191, 30] |
| 35 | hard coral alive | <img src="https://placehold.co/16x16/6de8bc/6de8bc.png" width="14" height="14"> [109, 232, 188] |
| 36 | hard coral bleached | <img src="https://placehold.co/16x16/46f795/46f795.png" width="14" height="14"> [70, 247, 149] |
| 37 | hard coral dead | <img src="https://placehold.co/16x16/720fcb/720fcb.png" width="14" height="14"> [114, 15, 203] |
| 38 | hard substrate | <img src="https://placehold.co/16x16/7d7d7d/7d7d7d.png" width="14" height="14"> [125, 125, 125] |
| 39 | human | <img src="https://placehold.co/16x16/ff0000/ff0000.png" width="14" height="14"> [255, 0, 0] |
| 40 | human-made structure | <img src="https://placehold.co/16x16/97ab50/97ab50.png" width="14" height="14"> [151, 171, 80] |
| 41 | leather coral alive | <img src="https://placehold.co/16x16/56b85e/56b85e.png" width="14" height="14"> [86, 184, 94] |
| 42 | lobophylliidae alive | <img src="https://placehold.co/16x16/c949ea/c949ea.png" width="14" height="14"> [201, 73, 234] |
| 43 | lobophylliidae bleached | <img src="https://placehold.co/16x16/fdbb4f/fdbb4f.png" width="14" height="14"> [253, 187, 79] |
| 44 | lobophylliidae dead | <img src="https://placehold.co/16x16/396adc/396adc.png" width="14" height="14"> [57, 106, 220] |
| 45 | massive alive | <img src="https://placehold.co/16x16/ec9615/ec9615.png" width="14" height="14"> [236, 150, 21] |
| 46 | massive bleached | <img src="https://placehold.co/16x16/fff8e4/fff8e4.png" width="14" height="14"> [255, 248, 228] |
| 47 | massive dead | <img src="https://placehold.co/16x16/865612/865612.png" width="14" height="14"> [134, 86, 18] |
| 48 | meandering coral alive | <img src="https://placehold.co/16x16/e6c100/e6c100.png" width="14" height="14"> [230, 193, 0] |
| 49 | meandering coral bleached | <img src="https://placehold.co/16x16/fbf3d8/fbf3d8.png" width="14" height="14"> [251, 243, 216] |
| 50 | meandering coral dead | <img src="https://placehold.co/16x16/77640e/77640e.png" width="14" height="14"> [119, 100, 14] |
| 51 | millepora | <img src="https://placehold.co/16x16/f49673/f49673.png" width="14" height="14"> [244, 150, 115] |
| 52 | millepora bleached | <img src="https://placehold.co/16x16/a4e234/a4e234.png" width="14" height="14"> [164, 226, 52] |
| 53 | millepora dead | <img src="https://placehold.co/16x16/7780ae/7780ae.png" width="14" height="14"> [119, 128, 174] |
| 54 | nephtheidae alive | <img src="https://placehold.co/16x16/846d5b/846d5b.png" width="14" height="14"> [132, 109, 91] |
| 55 | other animal | <img src="https://placehold.co/16x16/00ffff/00ffff.png" width="14" height="14"> [0, 255, 255] |
| 56 | other coral alive | <img src="https://placehold.co/16x16/e07677/e07677.png" width="14" height="14"> [224, 118, 119] |
| 57 | other coral bleached | <img src="https://placehold.co/16x16/fae0e1/fae0e1.png" width="14" height="14"> [250, 224, 225] |
| 58 | other coral dead | <img src="https://placehold.co/16x16/723c3d/723c3d.png" width="14" height="14"> [114, 60, 61] |
| 59 | pavona alive | <img src="https://placehold.co/16x16/0dbf66/0dbf66.png" width="14" height="14"> [13, 191, 102] |
| 60 | pavona bleached | <img src="https://placehold.co/16x16/76f982/76f982.png" width="14" height="14"> [118, 249, 130] |
| 61 | pavona dead | <img src="https://placehold.co/16x16/8d82af/8d82af.png" width="14" height="14"> [141, 130, 175] |
| 62 | pocillopora alive | <img src="https://placehold.co/16x16/ff9296/ff9296.png" width="14" height="14"> [255, 146, 150] |
| 63 | pocillopora bleached | <img src="https://placehold.co/16x16/fadbdc/fadbdc.png" width="14" height="14"> [250, 219, 220] |
| 64 | pocillopora dead | <img src="https://placehold.co/16x16/824242/824242.png" width="14" height="14"> [130, 66, 66] |
| 65 | porites alive | <img src="https://placehold.co/16x16/ffd147/ffd147.png" width="14" height="14"> [255, 209, 71] |
| 66 | porites bleached | <img src="https://placehold.co/16x16/d50a19/d50a19.png" width="14" height="14"> [213, 10, 25] |
| 67 | porites dead | <img src="https://placehold.co/16x16/2818f3/2818f3.png" width="14" height="14"> [40, 24, 243] |
| 68 | rubble | <img src="https://placehold.co/16x16/a19980/a19980.png" width="14" height="14"> [161, 153, 128] |
| 69 | sand | <img src="https://placehold.co/16x16/c2b280/c2b280.png" width="14" height="14"> [194, 178, 128] |
| 70 | sea cucumber | <img src="https://placehold.co/16x16/00e7ff/00e7ff.png" width="14" height="14"> [0, 231, 255] |
| 71 | sea urchin | <img src="https://placehold.co/16x16/008eff/008eff.png" width="14" height="14"> [0, 142, 255] |
| 72 | seagrass | <img src="https://placehold.co/16x16/7dde7d/7dde7d.png" width="14" height="14"> [125, 222, 125] |
| 73 | seriatopora alive | <img src="https://placehold.co/16x16/bf1306/bf1306.png" width="14" height="14"> [191, 19, 6] |
| 74 | seriatopora bleached | <img src="https://placehold.co/16x16/f759bf/f759bf.png" width="14" height="14"> [247, 89, 191] |
| 75 | seriatopora dead | <img src="https://placehold.co/16x16/4b8380/4b8380.png" width="14" height="14"> [75, 131, 128] |
| 76 | soft coral (malacalcyonacea) alive | <img src="https://placehold.co/16x16/939036/939036.png" width="14" height="14"> [147, 144, 54] |
| 77 | soft coral (malacalcyonacea) bleached | <img src="https://placehold.co/16x16/07e3f7/07e3f7.png" width="14" height="14"> [7, 227, 247] |
| 78 | sponge | <img src="https://placehold.co/16x16/f05050/f05050.png" width="14" height="14"> [240, 80, 80] |
| 79 | stylophora alive | <img src="https://placehold.co/16x16/ff6fc2/ff6fc2.png" width="14" height="14"> [255, 111, 194] |
| 80 | stylophora bleached | <img src="https://placehold.co/16x16/ffdcf1/ffdcf1.png" width="14" height="14"> [255, 220, 241] |
| 81 | stylophora dead | <img src="https://placehold.co/16x16/7b365e/7b365e.png" width="14" height="14"> [123, 54, 94] |
| 82 | table acropora alive | <img src="https://placehold.co/16x16/bd77ff/bd77ff.png" width="14" height="14"> [189, 119, 255] |
| 83 | table acropora bleached | <img src="https://placehold.co/16x16/f4e9fd/f4e9fd.png" width="14" height="14"> [244, 233, 253] |
| 84 | table acropora dead | <img src="https://placehold.co/16x16/553574/553574.png" width="14" height="14"> [85, 53, 116] |
| 85 | thin plate/encrusting coral alive | <img src="https://placehold.co/16x16/5b1cbe/5b1cbe.png" width="14" height="14"> [91, 28, 190] |
| 86 | thin plate/encrusting coral bleached | <img src="https://placehold.co/16x16/2ad139/2ad139.png" width="14" height="14"> [42, 209, 57] |
| 87 | thin plate/encrusting coral dead | <img src="https://placehold.co/16x16/0813f2/0813f2.png" width="14" height="14"> [8, 19, 242] |
| 88 | transect line | <img src="https://placehold.co/16x16/00ff00/00ff00.png" width="14" height="14"> [0, 255, 0] |
| 89 | transect tools | <img src="https://placehold.co/16x16/08cd0c/08cd0c.png" width="14" height="14"> [8, 205, 12] |
| 90 | trash | <img src="https://placehold.co/16x16/ff0086/ff0086.png" width="14" height="14"> [255, 0, 134] |
| 91 | turbinaria | <img src="https://placehold.co/16x16/e4ff77/e4ff77.png" width="14" height="14"> [228, 255, 119] |
| 92 | turbinaria dead | <img src="https://placehold.co/16x16/a91e46/a91e46.png" width="14" height="14"> [169, 30, 70] |
| 93 | water column | <img src="https://placehold.co/16x16/1f00a6/1f00a6.png" width="14" height="14"> [31, 0, 166] |
| 94 | water surface | <img src="https://placehold.co/16x16/00fff8/00fff8.png" width="14" height="14"> [0, 255, 248] |
| 95 | xeniidae alive | <img src="https://placehold.co/16x16/8b8ea8/8b8ea8.png" width="14" height="14"> [139, 142, 168] |
{
"0": "unlabeled",
"1": "acanthaster planci",
"2": "acropora alive",
"3": "acropora bleached",
"4": "acropora dead",
"5": "algae covered brain coral",
"6": "algae covered branching coral",
"7": "algae covered lobophylliidae",
"8": "algae covered massive coral",
"9": "algae covered pocillopora",
"10": "algae covered porites",
"11": "algae covered substrate",
"12": "algae covered table acropora",
"13": "anemone",
"14": "background",
"15": "brain coral alive",
"16": "brain coral bleached",
"17": "brain coral dead",
"18": "branching coral alive",
"19": "branching coral bleached",
"20": "branching coral dead",
"21": "cirrhipathes",
"22": "dark",
"23": "dead giant clam",
"24": "feather worm",
"25": "fish",
"26": "fungiidae alive",
"27": "fungiidae bleached",
"28": "fungiidae dead",
"29": "galaxea alive",
"30": "galaxea bleached",
"31": "galaxea dead",
"32": "giant clam",
"33": "goniopora alive",
"34": "goniopora dead",
"35": "hard coral alive",
"36": "hard coral bleached",
"37": "hard coral dead",
"38": "hard substrate",
"39": "human",
"40": "human-made structure",
"41": "leather coral alive",
"42": "lobophylliidae alive",
"43": "lobophylliidae bleached",
"44": "lobophylliidae dead",
"45": "massive alive",
"46": "massive bleached",
"47": "massive dead",
"48": "meandering coral alive",
"49": "meandering coral bleached",
"50": "meandering coral dead",
"51": "millepora",
"52": "millepora bleached",
"53": "millepora dead",
"54": "nephtheidae alive",
"55": "other animal",
"56": "other coral alive",
"57": "other coral bleached",
"58": "other coral dead",
"59": "pavona alive",
"60": "pavona bleached",
"61": "pavona dead",
"62": "pocillopora alive",
"63": "pocillopora bleached",
"64": "pocillopora dead",
"65": "porites alive",
"66": "porites bleached",
"67": "porites dead",
"68": "rubble",
"69": "sand",
"70": "sea cucumber",
"71": "sea urchin",
"72": "seagrass",
"73": "seriatopora alive",
"74": "seriatopora bleached",
"75": "seriatopora dead",
"76": "soft coral (malacalcyonacea) alive",
"77": "soft coral (malacalcyonacea) bleached",
"78": "sponge",
"79": "stylophora alive",
"80": "stylophora bleached",
"81": "stylophora dead",
"82": "table acropora alive",
"83": "table acropora bleached",
"84": "table acropora dead",
"85": "thin plate/encrusting coral alive",
"86": "thin plate/encrusting coral bleached",
"87": "thin plate/encrusting coral dead",
"88": "transect line",
"89": "transect tools",
"90": "trash",
"91": "turbinaria",
"92": "turbinaria dead",
"93": "water column",
"94": "water surface",
"95": "xeniidae alive"
}
</details>
<details>
<summary><b><code>label2color</code> (class name → RGB)</b></summary>
{
"acanthaster planci": [
179,
245,
234
],
"acropora alive": [
236,
128,
255
],
"acropora bleached": [
255,
236,
254
],
"acropora dead": [
118,
63,
128
],
"algae covered brain coral": [
208,
207,
58
],
"algae covered branching coral": [
6,
71,
245
],
"algae covered lobophylliidae": [
50,
208,
130
],
"algae covered massive coral": [
67,
182,
208
],
"algae covered pocillopora": [
225,
109,
77
],
"algae covered porites": [
173,
81,
130
],
"algae covered substrate": [
125,
163,
125
],
"algae covered table acropora": [
48,
162,
221
],
"anemone": [
0,
255,
189
],
"background": [
29,
162,
216
],
"brain coral alive": [
114,
78,
120
],
"brain coral bleached": [
219,
67,
238
],
"brain coral dead": [
78,
172,
99
],
"branching coral alive": [
226,
91,
157
],
"branching coral bleached": [
252,
231,
240
],
"branching coral dead": [
123,
50,
86
],
"cirrhipathes": [
6,
189,
22
],
"dark": [
31,
31,
31
],
"dead giant clam": [
89,
155,
134
],
"feather worm": [
159,
186,
38
],
"fish": [
255,
255,
0
],
"fungiidae alive": [
100,
133,
248
],
"fungiidae bleached": [
200,
138,
27
],
"fungiidae dead": [
3,
92,
44
],
"galaxea alive": [
96,
188,
108
],
"galaxea bleached": [
67,
243,
60
],
"galaxea dead": [
176,
225,
11
],
"giant clam": [
189,
255,
234
],
"goniopora alive": [
181,
5,
120
],
"goniopora dead": [
179,
191,
30
],
"hard coral alive": [
109,
232,
188
],
"hard coral bleached": [
70,
247,
149
],
"hard coral dead": [
114,
15,
203
],
"hard substrate": [
125,
125,
125
],
"human": [
255,
0,
0
],
"human-made structure": [
151,
171,
80
],
"leather coral alive": [
86,
184,
94
],
"lobophylliidae alive": [
201,
73,
234
],
"lobophylliidae bleached": [
253,
187,
79
],
"lobophylliidae dead": [
57,
106,
220
],
"massive alive": [
236,
150,
21
],
"massive bleached": [
255,
248,
228
],
"massive dead": [
134,
86,
18
],
"meandering coral alive": [
230,
193,
0
],
"meandering coral bleached": [
251,
243,
216
],
"meandering coral dead": [
119,
100,
14
],
"millepora": [
244,
150,
115
],
"millepora bleached": [
164,
226,
52
],
"millepora dead": [
119,
128,
174
],
"nephtheidae alive": [
132,
109,
91
],
"other animal": [
0,
255,
255
],
"other coral alive": [
224,
118,
119
],
"other coral bleached": [
250,
224,
225
],
"other coral dead": [
114,
60,
61
],
"pavona alive": [
13,
191,
102
],
"pavona bleached": [
118,
249,
130
],
"pavona dead": [
141,
130,
175
],
"pocillopora alive": [
255,
146,
150
],
"pocillopora bleached": [
250,
219,
220
],
"pocillopora dead": [
130,
66,
66
],
"porites alive": [
255,
209,
71
],
"porites bleached": [
213,
10,
25
],
"porites dead": [
40,
24,
243
],
"rubble": [
161,
153,
128
],
"sand": [
194,
178,
128
],
"sea cucumber": [
0,
231,
255
],
"sea urchin": [
0,
142,
255
],
"seagrass": [
125,
222,
125
],
"seriatopora alive": [
191,
19,
6
],
"seriatopora bleached": [
247,
89,
191
],
"seriatopora dead": [
75,
131,
128
],
"soft coral (malacalcyonacea) alive": [
147,
144,
54
],
"soft coral (malacalcyonacea) bleached": [
7,
227,
247
],
"sponge": [
240,
80,
80
],
"stylophora alive": [
255,
111,
194
],
"stylophora bleached": [
255,
220,
241
],
"stylophora dead": [
123,
54,
94
],
"table acropora alive": [
189,
119,
255
],
"table acropora bleached": [
244,
233,
253
],
"table acropora dead": [
85,
53,
116
],
"thin plate/encrusting coral alive": [
91,
28,
190
],
"thin plate/encrusting coral bleached": [
42,
209,
57
],
"thin plate/encrusting coral dead": [
8,
19,
242
],
"transect line": [
0,
255,
0
],
"transect tools": [
8,
205,
12
],
"trash": [
255,
0,
134
],
"turbinaria": [
228,
255,
119
],
"turbinaria dead": [
169,
30,
70
],
"water column": [
31,
0,
166
],
"water surface": [
0,
255,
248
],
"xeniidae alive": [
139,
142,
168
]
}
</details>
If you find this model useful, please cite:
@inproceedings{sauder2026coralscapesv2,
title={CoralscapesV2: Panoptic and Fine-Grained Visual Scene Understanding in Coral Reefs},
author={Sauder, Jonathan and Ruckli, Thomas and Strodomskyt{\.e}, Gabriel{\.e} and Abdallah, Ibrahim Souleiman and Abdi, Rahma Hassan and Awaleh, Djama Goumaneh and Farah, Mohamed Houssein and Nour, Moustapha and Saad, Osama Sharhubil and Altaib, Mustafa Mohammed Khalafallah and Kteifan, Maysoon and Alsoqi, Farah and Zgool, Eyad and Al-Omari, Jafar and Gebreluel, Temesgen Gebremeskel and Abdulkerim, Zekaria Zekeria and Ghirmay, Meron and Beraki, Teklehaimanot and Tuia, Devis and Banc-Prandi, Guilhem},
booktitle={Proceedings of the European Conference on Computer Vision (ECCV) Workshop on Marine Vision},
year={2026}
}
The dataset builds on the original Coralscapes dataset:
@inproceedings{sauder2025coralscapes,
title={The Coralscapes Dataset: Semantic Scene Understanding in Coral Reefs},
author={Sauder, Jonathan and Domazetoski, Viktor and Banc-Prandi, Guilhem and Perna, Gabriela and Meibom, Anders and Tuia, Devis},
booktitle={Proceedings of the International Conference on Computer Vision (ICCV) Joint Workshop on Marine Vision},
year={2025}
}
Released under the Apache License 2.0.