sf3d-ios

SF3D-iOS — on-device image-to-3D for HaploAI

Two-file ONNX port of Stable Fast 3D (SF3D) for running image → 3D mesh entirely on an iPhone/iPad via ONNX Runtime. This is the successor to jc-builds/triposr-ios: same on-device shape (feed-forward image → triplane → decoder → marching cubes), but a 384×384 triplane instead of 64×64 — a 36× denser feature grid — plus SF3D's illumination-disentanglement (de-lit albedo), so reconstructions have sharper geometry and cleaner texture than TripoSR.

⚠️ License: Stability AI Community License (not MIT, unlike triposr-ios). Free for research, non-commercial, and commercial use only while your organization's annual revenue is under US $1,000,000. Commercial users must register at https://stability.ai/community-license and display "Powered by Stability AI". Over $1M revenue requires an enterprise license. See LICENSE.md and NOTICE. This is a derivative work — weights were converted to ONNX/fp16, not retrained.

Files

Verified: encoder ONNX↔torch parity max|Δ| = 4.4e-4 (rel 0.0014%); decoder parity 1e-6; full encoder→decoder→marching-cubes produces a watertight mesh (density crosses the 10.0 iso threshold as expected).

Multi-view

A feed-forward multi-view encoder (export_mv_onnx.py) is provided as a script but not shipped as weights here: SF3D was trained single-view, so fusing several views through its camera-conditioned path is architecturally supported but not quality-validated. For a validated multi-image reconstruction (+32.9% Chamfer over single-view), use jc-builds/haplosr-mv, which refines the triplane against posed views (ARKit gives the poses for free).

The decoder I/O contract is identical to triposr-ios ([1,N,120] → [1,N,4], planes concatenated in xy, xz, yz order, 40 channels each), so the existing TriplaneDecoder/marching-cubes code works unchanged. Only two constants differ from TripoSR and must be set per-model in the app:

On-device inference (matches HaploAI's ONNXRuntimeService)

1. Preprocess to [1,3,512,512], RGB in [0,1], background composited onto gray 0.5 (SF3D background_color).
2. sf3d_encoder.onnx → triplane [1,3,40,384,384].
3. For each 3D query point, bilinearly sample the 3 planes (align_corners=true, positions scaled (-0.87, 0.87) → (-1, 1)), concat to 120 dims.
4. sf3d_decoder.onnx → density + rgb.
5. Marching cubes at iso 10.0.

Reproduce

Scripts: export_onnx.py (single-view encoder + decoder) with sdpa_symbolic.py (memory-lean ONNX attention that fits the export in RAM), consolidate_fp16.py (→ single fp16 file, fp32 I/O), validate_e2e.py / validate_torch.py (full encoder→decoder→mesh check), and export_mv_onnx.py (multi-view encoder, for a higher-RAM machine).

Attribution

This Stability AI Model is licensed under the Stability AI Community License, Copyright © Stability AI Ltd. All Rights Reserved. Converted to ONNX (fp16 weights, fp32 I/O, DINOv2 positional embeddings frozen for a fixed 512×512 input) by jc-builds; the network weights are unchanged. Powered by Stability AI.