sallunet

SalluNet

A 15 MB background-removal model for people. 4.03M parameters, distilled from BiRefNet_HR-matting and trained on ~14k images for about $33 of serverless GPU.

Try it in your browser — runs on-device, nothing is uploaded.

Code, full method, side-by-side comparisons against rembg and RVM, and the complete record of failed experiments: https://github.com/salmannawaz786/sallunet

Files

Usage

import numpy as np, onnxruntime as ort
from PIL import Image
from huggingface_hub import hf_hub_download

path = hf_hub_download("salluu3432/sallunet", "sallunet_512.onnx")
sess = ort.InferenceSession(path, providers=["CPUExecutionProvider"])

img = Image.open("person.jpg").convert("RGB")
w, h = img.size

# The model is trained on squashed 512x512 squares — do not letterbox.
x = np.asarray(img.resize((512, 512), Image.BILINEAR), np.float32) / 255.0
x = np.transpose(x, (2, 0, 1))[None]

alpha = np.clip(sess.run(None, {sess.get_inputs()[0].name: x})[0][0, 0], 0, 1)
matte = Image.fromarray((alpha * 255).astype(np.uint8)).resize((w, h), Image.BILINEAR)

cutout = img.copy()
cutout.putalpha(matte)
cutout.save("cutout.png")

Input is NCHW float32 RGB in [0,1] at 512×512. Output is a single-channel alpha matte at 512×512.

Limitations

• People only. A car or a dog produces meaningless output.
• Still images. No temporal consistency — it flickers on video.
• Benchmarked in-domain. The eval split and 70% of training are both P3M, so real-world photos are harder than the numbers below suggest.
• Fine flyaway hair on busy backgrounds, heavy backlighting, and motion blur all degrade the matte.

---

Results

All models were run through the same evaluation code, at the same resolution, over the same 500 held-out images (P3M-10k / P3M_500_NP). Latency is ONNX Runtime on 4 CPU threads.

band_mad is mean absolute alpha error restricted to the soft transition band (0.05 < α < 0.95) — i.e. hair. It is the metric this project optimises for, because hair is what separates a matting model from a segmentation model.

SalluNet vs rembg: better on every metric, 10.9× smaller, and faster despite processing 2.6× more pixels per frame.

SalluNet vs RVM: 18% better hair and better edge structure, but RVM wins whole-image accuracy and is 2.1× faster. RVM is an extremely well-engineered model and this project does not beat it outright.

SalluNet vs its own teacher: better hair than the 220M model it was distilled from — because 70% of the training mix was P3M's human-drawn alphas, not teacher pseudo-labels. The student learned from people, not just from a model.

Caveats that belong next to the numbers

• This is an in-domain result. The eval split and 70% of training are both P3M. rembg, RVM and the teacher never saw it. The win is real but flattered.
• sad is not comparable to published P3M figures. SAD is a sum over pixels; these are measured at 512², published numbers usually at native resolution. Only the within-table comparison is meaningful.
• mad and sad are not reliable at this scale. A replicate run measured their run-to-run variance at 17%; band_mad and grad sit at ~2%. See What the numbers can and cannot say.
• Evaluation squashes images to square, which distorts portrait aspect ratios. Applied identically to every model, so the ranking holds, but it is a flaw.

---

How it works

Student — MobileNetV4-conv-small encoder, a 224-channel FPN decoder run to stride 2, and a shallow full-resolution detail branch that predicts a residual on top of the semantic prediction. Zero-initialised, so training starts from the semantic output and learns edges from there.

Why the split: locating a person needs receptive field and tolerates low resolution; resolving a hair edge needs resolution and almost no receptive field. One head cannot do both — predicting alpha from strided features alone produces the blobby silhouettes this project set out to avoid.

Loss — band-weighted L1 (transition pixels weighted 10×) + Sobel gradient loss + Laplacian pyramid loss + a direct term on the coarse head. Without the band weighting a model scores well by predicting a clean silhouette and ignoring hair entirely, which is exactly the failure mode being designed against.

Data — 13,764 training pairs:

COCO images were filtered (person 15–98% of frame, ≤2 people, min side 480px), pseudo-labelled, then gated by dilated COCO person polygons to remove held objects, and finally filtered again on teacher-confidence signals. 50% of candidates were rejected.

---

What the numbers can and cannot say

Seven variants were trained. Six scored worse than the one that shipped:

Rather than assume those differences were real, v1 was re-trained with an identical configuration (v1r) to measure the noise floor directly. The two runs differ only in random weight initialisation — the data pipeline is deterministic, since ShardDataset seeds from step, which is 0 on every fresh run.

The metrics have very different noise floors, and it changes which conclusions survive.

band_mad and grad are reliable to ~2%. mad and sad are not — two runs of the same configuration differ by 17% on them. The reason is structural: band_mad is normalised over the transition band only, while sad/mad are whole-image aggregates dominated by large flat regions where tiny per-pixel differences accumulate.

So:

• Conclusions that stand. The stride-2 decoder is genuinely the best configuration tested — the 20% band_mad gap to the stride-4 runs is 10× the noise floor. v1 is reproducible within 2%, so the headline is not a lucky draw. The 18% hair advantage over RVM is far outside noise.
• Conclusions that collapse. Every band_weight result. It "improved" mad 43% in the v4/v5 pair and made it 33% worse in the v1/v7 pair — both single runs on a metric with a ±17% floor, which is exactly why they disagreed. The mad gap to RVM (71%) is still real at 4× the floor, but nothing in this project measured a way to close it.

The broader lesson: several regressions in the table above were diagnosed, given mechanisms, and "fixed" before anyone checked whether the metric could resolve them. Measure the noise floor before interpreting a difference.

---

Things that were tried and did not work

Documented because the negative results were expensive and are more informative than the successes.

1024² fine-tuning made the model worse (band_mad 136 → 159 at 1024, → 261 at 512). Root cause: shards are stored at max 1024 on the long side, and the training crop takes 50–100% of that and resizes up to 1024. The model was being trained on 2× upsampled blur and learned to reproduce interpolation artifacts. Real high-resolution training needs higher-resolution sources; P3M originals are ~1000px and COCO ~640px, so the pixels do not exist.

INT8 quantization destroyed quality. Static QDQ was 1.6× faster but band_mad went 136 → 273, worse than rembg. Dynamic quantization was 2.6× slower — onnxruntime's dynamic path accelerates MatMul, which a convnet barely uses. Matting regresses a continuous alpha and all the quality lives in the soft transition band; INT8 quantises exactly that band into coarse levels. Classification tolerates this, matting does not.

Heavier augmentation was 43% worse. Rotation stamps artificial black corners into both image and alpha, and background replacement is only an approximation (the true foreground colour is unknown, so compositing reuses the observed pixel) — at high probability it teaches the model that hair always carries a colour fringe.

The first teacher choice was wrong and cost a full labelling pass ($3.24). BiRefNet (base) is a salient object segmenter, not a human matter: it happily included a pizza the subject was holding. A bake-off had been run, but it tested "does it segment the right object" and never tested "is the hair soft" — the property the entire project depends on. Testing the wrong thing is as expensive as not testing.

Two tuning experiments were wasted optimising a model that was already the best thing on the benchmark, because no baseline had been measured. band_mad 136 is meaningless without knowing that rembg scores 249 and the teacher 163. Measure baselines before tuning.

---

Reproducing

Stages are independently resumable — every unit of work writes a .done marker after its output is safely on disk, so an interrupted run re-scans and skips what finished. A crash costs at most one shard.

modal run stage1_fetch.py::main          # COCO person subset  (CPU, ~2 min)
modal run stage1b_p3m.py::main           # P3M-10k             (CPU, free)
modal run stage2_label.py::main          # teacher labelling   (H100, ~$2.20)
python launch.py v1                      # student training    (A100, ~$2.60)
modal run evaluate.py::compare           # held-out metrics
modal run baseline_eval.py::main         # rembg / RVM / teacher / control
modal run export_bench.py::main          # ONNX + CPU latency

GPU is used only where a neural network actually runs — labelling and training. Downloading, filtering, packing and compositing are CPU, parallel, and effectively free.

Total cost: ~$30, of which roughly $8 was avoidable (a discarded labelling pass, an overlong first training run, and experiments run below the noise floor).

---

Licence and provenance

Code: MIT. The model is distilled from BiRefNet_HR-matting (MIT) and trained partly on P3M-10k (MIT) — it is not trained from scratch, and that framing is deliberate. P3M is a privacy-preserving dataset with deliberately blurred faces; COCO supplies the unblurred faces in the mix.