inspyrenet-onnx

InSPyReNet ONNX (Swin-B 1024x1024)

An ONNX export of InSPyReNet (Incongruent Salient Object Detection) using the Swin-B transformer backbone at 1024x1024 resolution. This model outputs fine-grained alpha mattes for zero-shot background removal, hair matting, and salient object extraction.

The model file is self-contained (all initializers and weights are embedded directly in the 357 MB file). You can run it on CPU, CUDA, Apple Silicon via CoreML, or inside web browsers with WebGPU through ONNX Runtime.

Model Summary

Available Quantizations

Quickstart (Python)

Install onnxruntime and pillow:

pip install onnxruntime pillow numpy
# Or for GPU support:
# pip install onnxruntime-gpu pillow numpy

Run background removal in Python:

import numpy as np
import onnxruntime as ort
from PIL import Image

# 1. Load image and keep original dimensions
img = Image.open("input.jpg").convert("RGB")
orig_w, orig_h = img.size

# 2. Resize to 1024x1024 and normalize
resized = img.resize((1024, 1024), Image.Resampling.BILINEAR)
arr = np.array(resized, dtype=np.float32) / 255.0

mean = np.array([0.485, 0.456, 0.406], dtype=np.float32)
std = np.array([0.229, 0.224, 0.225], dtype=np.float32)
norm = (arr - mean) / std

# 3. Shape to [1, 3, 1024, 1024]
tensor = np.transpose(norm, (2, 0, 1))[np.newaxis, ...]

# 4. Run inference (use model_fp16.onnx, model_int8.onnx, or model_q4.onnx as needed)
session = ort.InferenceSession("model_fp16.onnx", providers=["CUDAExecutionProvider", "CPUExecutionProvider"])
out = session.run(None, {"input": tensor})[0]

# 5. Extract alpha and resize back to original size
alpha = (np.clip(out[0, 0], 0.0, 1.0) * 255.0).astype(np.uint8)
alpha_img = Image.fromarray(alpha, mode="L").resize((orig_w, orig_h), Image.Resampling.BILINEAR)

# 6. Save transparent cutout
cutout = img.convert("RGBA")
cutout.putalpha(alpha_img)
cutout.save("cutout.png")

CLI Usage

This repository includes a ready-to-use Python script in examples/infer.py:

# Process a single image to a transparent PNG
python examples/infer.py --image photo.jpg --output cutout.png

# Process a single image on CUDA
python examples/infer.py --image photo.jpg --provider cuda

# Save only the grayscale alpha matte mask
python examples/infer.py --image photo.jpg --mask-only --output mask.png

# Batch process a folder of images
python examples/infer.py --dir path/to/images --output-dir path/to/cutouts

Browser Usage (WebGPU & WASM)

You can run this model directly on client devices inside the browser using onnxruntime-web.

A complete demo application is provided in examples/index.html. To test it locally:

cd examples
python3 -m http.server 8000

Open http://localhost:8000 in your browser. You can load the model via WebGPU or WASM, drag and drop any image, view the live alpha mask, and download transparent PNG cutouts.

Minimal JavaScript snippet:

import * as ort from "onnxruntime-web";

// Initialize session with WebGPU
const session = await ort.InferenceSession.create("inspyrenet_swinb_1024.onnx", {
  executionProviders: ["webgpu"]
});

// Prepare Float32Array tensor with shape [1, 3, 1024, 1024]
// normalized with ImageNet mean/std
const feeds = { input: inputTensor };
const results = await session.run(feeds);
const alphaData = results.alpha.data; // Float32Array of 1024 * 1024 alpha values

Technical Notes

• Input resolution: The model expects an input tensor of 1x3x1024x1024. Non-square images should be resized to 1024x1024 for inference, and the resulting alpha mask resized back to the original image dimensions.
• Output range: The output values represent foreground probability between 0.0 (background) and 1.0 (foreground). Soft edges (such as hair or transparent objects) contain fractional values between 0.0 and 1.0.
• Hardware acceleration: On modern desktop GPUs (RTX series, Apple Silicon M-series), inference runs in 50 to 120 ms. On multi-core CPUs, inference typically completes in 500 to 1200 ms depending on thread count.

Upstream Research & Attribution

This model is derived from the research project:

• Paper: InSPyReNet: Incongruent Salient Object Detection (ACM MM 2023)
• Authors: Taehun Kim, Kunhee Kim, Joonyeong Lee, Dongmin Cha, Jiho Lee, Daijin Kim
• Upstream Code: github.com/plemeri/InSPyReNet

License

The files in this repository are licensed under the Apache 2.0 License. See LICENSE for details. Upstream research code for InSPyReNet is licensed under the MIT License.