MiDaS-DPT-Hybrid-ONNX

MiDaS DPT-Hybrid - ONNX (FP16)

FP16 ONNX export of Intel's DPT-Hybrid MiDaS checkpoint, built for Oku3D's depth registry and usable by any ONNX Runtime consumer.

Interface

input   pixel_values     float32  [1, 3, 384, 384]
output  predicted_depth  float32  [1, 384, 384]

The frame is normalized with a mean and standard deviation of 0.5 per channel, not with the ImageNet statistics. That is what MiDaS selects for every DPT-architecture model and what Intel's processor config for this checkpoint carries; feeding ImageNet values instead costs several percent of the depth spread and nothing reports an error. The graph itself does not normalize, so the transform is the consumer's to apply.

The size is fixed at 384: this checkpoint's position embeddings are tied to that grid, and the exporter specializes the shape rather than inventing an interpolation the weights were never trained for.

The output is relative inverse depth, near = high - the MiDaS convention, which needs no inversion for stereo warping. Weights are FP16 while the boundary stays FP32, so a consumer hands over what it already has.

Accuracy

Mean absolute deviation from the PyTorch checkpoint over the four example images, as a percentage of the reference map's own depth spread:

Usage

import numpy as np, onnxruntime as ort

session = ort.InferenceSession(
    "midas-dpt-hybrid_fp16_opset20_384x384_optimized.onnx",
    providers=["DmlExecutionProvider", "CPUExecutionProvider"],
)
# pixels in [0, 1], then (value - 0.5) / 0.5
disparity = session.run(None, {"pixel_values": tensor})[0]  # [1, 384, 384]

License

Apache-2.0, inherited from the checkpoint; see LICENSE. Upstream: Vision Transformers for Dense Prediction, Ranftl et al., 2021, and isl-org/MiDaS.

Core ML build for macOS

The coreml/ folder holds this model rewritten so ONNX Runtime's Core ML execution provider takes the whole graph on a Mac - Neural Engine, GPU and CPU - instead of handing parts of it to the CPU. Oku3D uses it on macOS. Core ML runs only standard ONNX operators, in float16, so:

• coreml/midas-dpt-hybrid_fp16_opset20_384x384_coreml.onnx: FastGelu expanded; Expand written as a broadcasting Add; normalization inputs past float16's range scaled down.

The weights are this repository's; only the graph around them changes. Checked against the original model on ONNX Runtime's CPU provider, the depth maps agree at a correlation of 0.9999 or better. Same license as the rest of this repository.