Sharp-ONNX-HighPerf

SHARP - ONNX, re-exported and optimized

One photograph in, a 3D Gaussian scene out, in a single forward pass. This is Apple's SHARP re-exported from the PyTorch weights at opset 21: 1.13 s per image and a 6.7 GiB peak on the hardware below, 2422 nodes, float16 end to end, pure ai.onnx with no contrib operators.

[!IMPORTANT] Research use only. The Apple Machine Learning Research Model License grants use "exclusively for Research Purposes" and states that this "does not include any commercial exploitation, product development or use in any commercial product or service". The grant is revocable. Changing the file format changes none of that. Independent, unofficial conversion - not affiliated with or endorsed by Apple.

Examples

Each loop is one photograph turned into Gaussians, then a camera move rendered from them: a 10-degree lateral sway with a slight push in, 96 frames at 25 fps.

The input for each is examples/sampleN/source.jpg.

What this build gives you

An ONNX build of SHARP already exists, from Kyle Pearson, and this one reuses its traceable wrapper (see Acknowledgements). It is re-traced from the PyTorch weights rather than converted from that graph:

Measured on three images, comparing every Gaussian of each against pearsonkyle/Sharp-onnx, the mean absolute deviation per tensor stays between 0.011 and 0.280 % (positions), 0.031 and 0.100 % (scales), 0.039 and 0.182 % (quaternions), 0.013 and 0.034 % (colours), and 0.132 and 0.687 % (opacities) of that tensor's range - float16 boundary rounding.

A 12 GiB card keeps 5.5 GiB free, a 10 GiB card 3.5 GiB, an 8 GiB card 1.4 GiB. pearsonkyle/Sharp-onnx needed a 16 GiB card.

Specifications

Measured on an AMD Radeon RX 7900 XTX, ONNX Runtime 1.23, WebGPU plugin execution provider, Linux. DirectML was not measured, and WebGPU EP kernels come from the driver - measure on your own target before quoting a number. There is no 4-bit variant: one was built and rejected, 3.6x slower for 0.9 GiB less peak.

Quick start

import numpy as np, onnxruntime as ort
from PIL import Image

sess = ort.InferenceSession("sharp_1536x1536_bs1_fp16_opset21_optimized.onnx",
                            providers=["CPUExecutionProvider"])

# Square-crop first - a non-square source comes back distorted.
img = Image.open("photo.jpg").convert("RGB")
s = min(img.size)
img = img.crop(((img.width - s) // 2, (img.height - s) // 2,
                (img.width + s) // 2, (img.height + s) // 2)).resize((1536, 1536), Image.BILINEAR)
x = np.transpose(np.asarray(img, np.float32) / 255.0, (2, 0, 1))[None].astype(np.float16)

means, scales, quats, colors, opac = sess.run(
    None, {"image": x, "disparity_factor": np.array([1.0], np.float16)})

# Projective -> camera space. focal_ndc = 2 * f / w; f = image width here.
m = means[0].astype(np.float64)
z = m[:, 2]; k = 1.0 / (z.min() * 2.0)
xyz = np.stack([m[:, 0] * k, m[:, 1] * k, z / z.min()], -1)

Writing a PLY in the INRIA layout additionally needs the sRGB transfer plus the SH DC encoding (c - 0.5) / 0.28209479177387814 on the colours, an inverse sigmoid on the opacities, and log() on the scales - the graph emits all three already activated.

Three things that will bite you

The positions are projective, not camera space. Divide the first two components by focal_ndc = 2 * f / w. Feeding the raw values to a renderer produces a plausible-looking scene with wrong geometry, and nothing errors.

The input is a fixed 1536x1536 square and cannot be re-exported smaller. Depth Pro's patch pyramid concatenates 384-sized tensors, so a 1152 or 768 trace fails inside torch.cat. Crop or letterbox to square before resizing.

The assumed lens sets the shape of the scene, not just its scale. disparity_factor is focal_length / image_width; the reference pipeline assumes a 30 mm equivalent when EXIF says nothing. A portrait built at 30 mm when it was taken at 85 comes out pressed flat. If your image has no EXIF focal length, ask the user rather than defaulting silently.

Acknowledgements

• Apple Machine Learning Research for SHARP - the model, the weights, and the paper.
• Kyle Pearson (pearsonkyle/Sharp-onnx) for the first ONNX conversion. This build reuses its SharpModelTraceable wrapper, its output names and its checkpoint loader, and its inference_onnx.py is where the projective-coordinate convention documented above was worked out. That repository carries an apple-amlr tag covering its scripts as well as the weights.
• The example renders use an independent implementation of the EWA splatting formulation. No code from the INRIA reference implementation of 3D Gaussian Splatting is used anywhere here.

Citation

@inproceedings{Sharp2025:arxiv,
  title     = {Sharp Monocular View Synthesis in Less Than a Second},
  author    = {Lars Mescheder and Wei Dong and Shiwei Li and Xuyang Bai and Marcel Santos and
               Peiyun Hu and Bruno Lecouat and Mingmin Zhen and Ama\"{e}l Delaunoy and Tian Fang and
               Yanghai Tsin and Stephan R. Richter and Vladlen Koltun},
  journal   = {arXiv preprint arXiv:2512.10685},
  year      = {2025},
  url       = {https://arxiv.org/abs/2512.10685},
}