BiRefNet_dynamic-1024x1024-ONNX-CUDA

<h1 align="center">Bilateral Reference for High-Resolution Dichotomous Image Segmentation</h1> <div align="center"> <a href="https://scholar.google.com/citations?user=TZRzWOsAAAAJ" target="_blank"><strong>Peng Zheng</strong></a>^1,4,5,6,&thinsp; <a href="https://scholar.google.com/citations?user=0uPb8MMAAAAJ" target="_blank"><strong>Dehong Gao</strong></a>²,&thinsp; <a href="https://scholar.google.com/citations?user=kakwJ5QAAAAJ" target="_blank"><strong>Deng-Ping Fan</strong></a>^1*,&thinsp; <a href="https://scholar.google.com/citations?user=9cMQrVsAAAAJ" target="_blank"><strong>Li Liu</strong></a>³,&thinsp; <a href="https://scholar.google.com/citations?user=qQP6WXIAAAAJ" target="_blank"><strong>Jorma Laaksonen</strong></a>⁴,&thinsp; <a href="https://scholar.google.com/citations?user=pw_0Z_UAAAAJ" target="_blank"><strong>Wanli Ouyang</strong></a>⁵,&thinsp; <a href="https://scholar.google.com/citations?user=stFCYOAAAAAJ" target="_blank"><strong>Nicu Sebe</strong></a>⁶ </div> <div align="center"> ¹Nankai University&ensp; ²Northwestern Polytechnical University&ensp; ³National University of Defense Technology&ensp; ⁴Aalto University&ensp; ⁵Shanghai AI Laboratory&ensp; ⁶University of Trento </div>

This repository provides FP16 ONNX weights for ZhengPeng7/BiRefNet_dynamic, exported for ONNX Runtime CUDA and NVIDIA Triton. For Transformers.js and WebGPU, use onnx-community/BiRefNet_dynamic-1024x1024-ONNX.

For more information about BiRefNet, see the official repository.

Model Details

The batch axis is dynamic. The spatial input size is fixed at 1024 × 1024, and the output spatial size follows the input. ALPHA is a foreground probability in [0, 1]; sigmoid is included in the graph.

The model uses ONNX opset 19 and standard ONNX DeformConv operators. Use a CUDA-enabled ONNX Runtime build that supports FP16 DeformConv.

Usage (ONNX Runtime CUDA)

Install the dependencies:

pip install huggingface_hub numpy onnxruntime-gpu pillow

Run inference and save the foreground mask:

import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download
from PIL import Image

model_path = hf_hub_download(
    "onnx-community/BiRefNet_dynamic-1024x1024-ONNX-CUDA",
    "onnx/model.onnx",
)

image = Image.open("input.png").convert("RGB")
resized = image.resize((1024, 1024), Image.Resampling.BILINEAR)

pixel_values = np.asarray(resized, dtype=np.float32) / 255.0
mean = np.asarray([0.485, 0.456, 0.406], dtype=np.float32)
std = np.asarray([0.229, 0.224, 0.225], dtype=np.float32)
pixel_values = (pixel_values - mean) / std
pixel_values = np.transpose(pixel_values, (2, 0, 1))[None].astype(np.float16)

session = ort.InferenceSession(
    model_path,
    providers=["CUDAExecutionProvider"],
)
alpha = session.run(["ALPHA"], {"IMAGE": pixel_values})[0][0, 0]

mask = Image.fromarray((np.clip(alpha, 0, 1) * 255).astype(np.uint8))
mask.resize(image.size, Image.Resampling.BILINEAR).save("mask.png")

Do not apply sigmoid to the output again.

Usage (NVIDIA Triton)

Place the ONNX file at birefnet_dynamic/1/model.onnx in the Triton model repository and use:

name: "birefnet_dynamic"
platform: "onnxruntime_onnx"
max_batch_size: 1

input [
  { name: "IMAGE" data_type: TYPE_FP16 dims: [ 3, 1024, 1024 ] }
]

output [
  { name: "ALPHA" data_type: TYPE_FP16 dims: [ 1, -1, -1 ] }
]

instance_group [
  { kind: KIND_GPU count: 1 }
]

Configure dynamic batching and queue limits according to the serving workload.

Conversion and Validation

• Source: ZhengPeng7/BiRefNet_dynamic at revision 280306042f57b7a33854319da62fd86aaa89ec4c
• ONNX IR version: 9
• ONNX opset: 19
• Precision: FP16 weights, input, and output
• External data: none
• ONNX checker: passed
• ONNX Runtime CUDA FP16 vs. source PyTorch FP16:
  • Mean absolute error: 1.4735e-5
  • Maximum absolute error: 0.004486
• NVIDIA Triton 26.08 end-to-end validation: passed

Artifact details and checksums are available in manifest.json.

Limitations

• The model accepts RGB tensors only; image decoding, orientation handling, resizing, and alpha compositing are application responsibilities.
• Despite the upstream model variant name, this ONNX export does not accept arbitrary spatial input sizes.
• CPU and browser execution providers are not validation targets for this artifact.

Citation

@article{BiRefNet,
  title={Bilateral Reference for High-Resolution Dichotomous Image Segmentation},
  author={Zheng, Peng and Gao, Dehong and Fan, Deng-Ping and Liu, Li and Laaksonen, Jorma and Ouyang, Wanli and Sebe, Nicu},
  journal={CAAI Artificial Intelligence Research},
  year={2024}
}