birefnet-onnx-int8

BiRefNet ONNX INT8

Dynamic INT8 quantisation of onnx-community/BiRefNet_dynamic-1024x1024-ONNX (commit 6defd3a19cec21042b178108815c9c4508d09867), itself an export of ZhengPeng7/BiRefNet_dynamic. Published because no INT8 BiRefNet exists publicly.

Against the 927 MiB float model it came from: 3.1x smaller, mattes unchanged.

Contract

Read off the graph, not off a model card — the CUDA export of the same base disagrees with this one about its own output.

• input input_image, [1, 3, 1024, 1024] f32. The spatial axes are fixed despite the "dynamic" in the name it inherits: that word describes the base model's training, not the export. Feeding any other size is a shape error.
• output output_image, [1, 1, 1024, 1024] f32, raw logits — measured range on real images reaches [-27, +80]. Apply a sigmoid exactly once.
• preprocessing: x/255, then ImageNet normalisation, mean [0.485, 0.456, 0.406] and std [0.229, 0.224, 0.225]. A squash to the square, not a letterbox.

What quantising cost

Nothing measurable. Fraction of partial-alpha pixels — the figure that separates a clean cutout from a holed one — before and after, on the same four pictures:

Coverage moved by at most 0.1 points, and the cutouts are indistinguishable over a chequerboard.

It does not make it faster. 13–15 s per pass on CPU against the float model's 14–21 s, same machine. The saving is the download.

How it was made

Two passes, in this order. The second alone makes the file bigger.

python fold_identity.py model.onnx folded.onnx   # 99 Identity nodes folded
python quantize.py folded.onnx birefnet_int8.onnx quint8

onnxruntime's dynamic quantiser only converts a MatMul whose B side is an initializer wired straight into the node. In this export 99 of the largest weights — the 37.7 MB attention matrices among them — reach their MatMul through an Identity, so all of them were left in float while quantised copies of everything smaller were added beside them: 927 MB in, 1,043 MB out. Folding those Identity nodes first gives 301 MB.

Scripts: fold_identity.py, quantize.py, in this repository.