birefnet-lite-1024-webgpu

BiRefNet-lite 1024 (ONNX, WebGPU-runnable)

A structurally patched ONNX export of BiRefNet-lite at 1024x1024 that runs on onnxruntime-web's WebGPU execution provider. The upstream export does not: it falls back to the CPU EP on most of its graph and dies of std::bad_alloc.

If you have tried to run BiRefNet at 1024 in a browser and hit an out-of-memory error, this repo is the fix, and the section below explains what was actually wrong. The usual workaround is to drop to a 512 export. That works and it is a reasonable trade — but the OOM is not caused by the resolution, and you do not have to pay it.

The weights are unchanged. Only the graph topology was rewritten, and both rewrites are proved bit-identical to the original on the CPU EP (max|diff| = 0.000e+00 over the test set).

Production use

This graph is the best quality tier in Backgroundless, a background remover where the model runs entirely in the visitor's browser — no upload, no server round-trip, no account. Both tiers are free to use, so the 1024 graph is served to anyone whose adapter can run it.

Quick start (transformers.js)

import { AutoModel } from '@huggingface/transformers';

const model = await AutoModel.from_pretrained('jiabins0303/birefnet-lite-1024-webgpu', {
  device: 'webgpu',
  dtype: 'fp32',                  // the file is ALREADY fp16; this stops the
  model_file_name: 'model_fp16',  // library appending a dtype suffix
});

Requires an adapter reporting maxStorageBuffersPerShaderStage >= 8. The graph emits logits, not an image — apply sigmoid, do not read it as 0-255.

Why the upstream export cannot run on WebGPU

Two independent blockers, which have to be fixed in this order because the first one masks the second:

1. 59 Split nodes with 32 outputs each. WebGPU's maxStorageBuffersPerShaderStage is 8 by spec (10 on many adapters), so these never compile. Rewritten into trees of <=6 outputs.
2. 80 GatherND nodes. deform_conv2d has no ONNX operator, so the exporter emulates it with GatherND, and in the onnxruntime-web build we tested (August 2026) the WebGPU EP did not take those nodes: all 80 ran on the CPU EP (the verbose log adds a MemcpyFromHost after each), which meant materialising a [1,1,64,49,256,256] fp16 im2col buffer — 392MB, several live at once — on ORT Web's 32-bit wasm heap. That is the std::bad_alloc; the resolution itself was never the problem. Rewritten to Gather plus int32 index arithmetic, and variadic Sum to binary Add.

Host<->device copies drop from 100 to 80 and the graph runs end to end on WebGPU.

Reproduce it: the patch scripts

The three scripts that produced this graph are in scripts/ (MIT, like the rest of this repo). They are topology-only rewrites and should work on other BiRefNet exports that hit the same two blockers.

pip install onnx onnxruntime numpy pillow
python scripts/patch_split.py  model.onnx step1.onnx       # Split trees, <= 6 outputs
python scripts/patch_deform.py step1.onnx patched.onnx     # GatherND -> Gather, Sum -> Add chain
python scripts/verify_patch.py model.onnx patched.onnx a.jpg b.jpg   # must report EXACT

patch_deform.py only rewrites the exact deform-conv GatherND pattern and refuses to rewrite one unless it can see a clamp upstream, so an index cannot silently wrap. verify_patch.py runs both graphs on the CPU EP; a structural rewrite should be bit-identical, and it says so if it is not.

Two things that are easy to get wrong if you redo this:

• Cast to int32 LATE. Doing the index arithmetic in int32 doubles the CPU<->GPU crossings (180 copies / 9.1s vs 80 / 3.4s). Do the maths in int64 and cast once, immediately before the Gather.
• Use a sequential Add chain, not a balanced tree. fp16 addition is not associative; a balanced tree changes the summation order and breaks bit-identity with the original graph.

Measurements

Against BiRefNet-lite-512 on the same 30-image set, same compositor, same metric (edgeSharpness().gradient):

+45% edge gradient for ~4x the time. It also recovers subjects 512 misses entirely — one test image returns an empty matte at 512 and 18.4% foreground here.

⚠️ Read the provenance before quoting these. The quality columns and the timing column come from different runs. The 1024 gradient figures were measured in Node on the CPU EP, not on the WebGPU path the timings describe; graph bit-identity was proven CPU-vs-CPU for the rewrite, which is not the same as proving CPU and WebGPU produce identical output. And one flag is still open: this graph's product softEdgeRatio is 0.01030 against a 0.01574 control — 35% lower — which under an anti-aliasing guard means part of the gradient win could be a harder, less anti-aliased edge rather than a better one. Band width (9.15) and the crops both support a genuinely tighter edge, but it is not fully cleared. Treat +45% as a strong indication, not a settled result, and re-measure on your own path.

Which one should you use

Not always this one.

• 512 is the better default for interactive, one-image-at-a-time UIs, for batch jobs where latency multiplies by the file count, and for wide device support. 0.85s is a different product feel from 3.4s.
• 1024 (this) is worth it when edge quality is the point — hair, fur, fabric, fine product edges — or when 512 misses the subject entirely.

Shipping both and letting the user choose is a reasonable design; that is what Backgroundless does. Only one graph should be resident at a time — switching disposes and recompiles.

Licensing

MIT — and unusually for a model, that covers the weights, not just the surrounding code. No non-commercial clause, no separate model agreement to accept, and nothing in it places any condition on the images you produce with it.

Worth checking on whatever you end up picking: licensing on browser matting models varies a lot, and the gap between a project's code licence and its weights licence is where people get caught. BRIA's RMBG-1.4 card, for example, states the model is "available as a source-available model for non-commercial use" — and an MIT-licensed wrapper repo around it does not change that, because the wrapper's licence covers the wrapper. Read the weights licence, not the repo badge.

⚠️ One upstream caveat, in fairness. BiRefNet_lite lists DIS5K among its training sets, and DIS5K's own terms restrict commercial use of the dataset. That is an open question for every model trained on it, this one included, and it is not something an MIT grant downstream can resolve. What MIT does give you here is a clean, explicit grant from the model's author covering the weights themselves — which is more than several alternatives offer, but it is not a warranty about the training data.

Credits and licence

• Original model: ZhengPeng7/BiRefNet (MIT) — Peng Zheng et al.
• ONNX export this was derived from: onnx-community/BiRefNet_lite-ONNX
• studioludens/birefnet-lite-512 — Studio Ludens' 512 re-export, and the export method (PyTorch 2.0.1 + Kazuhito00's patched deform-conv exporter) that made a working browser BiRefNet possible at all. Their model card is also where the "1024-way concatenations" problem is documented, which is what sent us looking at the decoder in the first place. Backgroundless ships their 512 export as its default tier — this 1024 graph is the opt-in one.
• Kazuhito00's deform-conv2d-onnx-exporter, via the above.
• Graph surgery: this repo. MIT, same as upstream.

If you use this, cite the BiRefNet authors — the model is theirs. The only thing contributed here is the graph rewrite.

Provenance

onnx/model_fp16.onnx sha256 4059896039dfccb0f15b9080ff06d11d90e499449bb045e797055eb8901cf5f4

Reproduce with patch_split.py then patch_deform.py, and verify with verify_patch.py (correlation gate plus max-abs-diff against the unpatched graph on the CPU EP).