mobilesam

MobileSAM: Optimized for AMD ROCm

MobileSAM is a lightweight, promptable segmentation model — a distilled Segment Anything Model (SAM) that predicts a pixel-level mask for any object from a single point prompt. This repository packages inference for promptable instance segmentation using ONNX Runtime, exported and validated for AMD ROCm so it runs efficiently on AMD GPUs, CPUs, and NPUs.

This is based on the implementation of MobileSAM found here. This repository contains configurations and scripts optimized for AMD® ROCm™ platforms. You can use the mobilesam AMD scripts to reproduce results or export with custom configurations. More details on model performance can be found here.

---

Task Overview

Task: Promptable instance segmentation (point prompt → mask)

Dataset: SA-1B subset (10 images, per-mask point-prompt annotations)

Output metrics: mIoU, Mean Mask IoU, Median Mask IoU, IoU Std Dev, Min IoU, Max IoU

---

AMD ROCm Optimization

This model export has been adapted and validated for AMD Instinct™ / Radeon™ GPUs running ROCm, as well as AMD CPUs and AMD Ryzen AI NPUs. Key points:

• Validated backends: ONNX Runtime across CPU (FP32), GPU (MIGraphX execution provider — FP32/FP16 for benchmarking, FP32/FP16/BF16/INT8 for profiling and evaluation), and NPU (VitisAI execution provider, auto-quantized internally).
• The pipeline is a two-stage ONNX graph — a TinyViT image encoder (once per image) feeding a prompt/mask decoder (once per point).
• No code changes required versus the upstream MobileSAM implementation — only environment/runtime configuration differs.
• On GPU, the MIGraphX execution provider writes the TinyViT encoder output in NHWC memory order; the evaluator reinterprets it back to NCHW so GPU accuracy matches CPU.

---

Getting Started

For setup instructions, evaluation scripts, and custom configuration options, see the mobilesam on GitHub.

---

Model Details

Model Type: Promptable instance segmentation — distilled Segment Anything Model with a TinyViT encoder

Base Model: MobileSAM (pinned commit b01a9ccef3b9e10b099b544efe004d0871802c3b)

Model Stats:

• Two-stage pipeline: TinyViT image encoder + prompt/mask decoder
• Encoder input: (1, 3, 1024, 1024) float32; encoder output (image_embeddings): (1, 256, 64, 64) float32
• Decoder input (point_coords): (1, 1, 2) float32; (point_labels): (1, 1) float32
• Decoder output (low_res_masks): (1, 1, 256, 256) float32; (iou_predictions): (1, 1) float32
• Precision tested: FP32 (CPU); FP32, FP16, BF16, INT8 (GPU); auto-quantized (NPU)

---

Accuracy Pipeline

Higher mIoU means the model's predicted masks overlap more closely with the ground-truth annotations — 1.0 would be pixel-perfect agreement, 0.0 means no overlap. In practice, values above ~0.5 for mIoU on SA-1B are considered strong for prompted segmentation.

Metrics Explained

Accuracy Results

Full Dataset Evaluation (SA-1B subset, 10 images) — filled from runs/quality_<device>_<precision>.json; run make metrics to refresh:

<!-- accuracy-table-start --> <!-- accuracy-table-end -->

---

Dig Deeper

Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?

📂 View the full project on GitHub

The GitHub repository includes:

• Setup and prerequisites for ROCm environments
• Scripts for CPU, GPU, and NPU runners (encoder + decoder benchmarking/profiling)
• SA-1B dataset staging and mIoU evaluation pipeline
• Benchmarking and reproduction instructions