face_landmarks_detector-ONNX

MediaPipe face landmarks detector (478 points), ONNX

This is the face landmarks model from Google's MediaPipe Face Landmarker, converted from TFLite to ONNX so it runs anywhere onnxruntime does, with no MediaPipe or TensorFlow needed. The weights are Google's and unchanged. Only the file format is different. See the original: MediaPipe Face Landmarker and its model card.

It predicts 478 3-D face landmarks, the 468-point face mesh plus 10 iris points, in a square crop around one face.

Usage

• Input input_12: float32 [1, 256, 256, 3], NHWC, RGB, values in [0, 1]. A square crop around the face, ideally rolled so the eyes are level. MediaPipe gets the crop from its face detector, or from the previous frame's landmarks.
• Output Identity: [1, 1, 1, 1434], the 478 × (x, y, z) landmarks in crop pixels (0..256).
• Output Identity_1: [1, 1, 1, 1], the face presence logit. Apply a sigmoid; MediaPipe keeps the face above about 0.5.
• Output Identity_2: [1, 1], an auxiliary output of the original graph.

import math, numpy as np, onnxruntime as ort
from PIL import Image

sess = ort.InferenceSession("onnx/model.onnx", providers=["CPUExecutionProvider"])
img = Image.open("portrait.jpg").convert("RGB")
cx, cy, size = 512, 400, 420                 # a square around the face, e.g. from a face detector
crop = img.crop((cx - size / 2, cy - size / 2, cx + size / 2, cy + size / 2)).resize((256, 256))
out = sess.run(None, {"input_12": (np.asarray(crop, np.float32) / 255.0)[None]})
pts = out[0].reshape(478, 3)
presence = 1 / (1 + math.exp(-float(out[1].ravel()[0])))
xy = pts[:, :2] * (size / 256.0) + [cx - size / 2, cy - size / 2]   # back to image pixels
# mesh indices: 33/263 eye corners, 468-472 and 473-477 the two irises, 13/14 inner lips

Refining the crop once or twice from the first pass's own points (box from the points, roll from the eye line) is how MediaPipe tracks, and it tightens the result noticeably. This takes about 5–15 ms per pass on a laptop CPU (Intel Core Ultra 9 185H, onnxruntime 1.30).

Conversion

The source is face_landmarks_detector.tflite from the versioned Face Landmarker bundle:

• bundle: https://storage.googleapis.com/mediapipe-models/face_landmarker/face_landmarker/float16/1/face_landmarker.task
• bundle sha256: 64184e229b263107bc2b804c6625db1341ff2bb731874b0bcc2fe6544e0bc9ff
• the TFLite inside: sha256 c7d54204ce0448474c7f3fa9af494787c0965cbdd6f20fc72867e43046bd43d5

The .task file is a zip archive; unzip it to get the TFLite.

pip install tf2onnx==1.17.0 tensorflow   # conversion only
python -m tf2onnx.convert --tflite face_landmarks_detector.tflite --opset 17 --output model.onnx

The result was checked against the TFLite model run through TensorFlow Lite on three photographs. The largest absolute difference is 0.00024 px on the landmarks and 2.2e-5 on the presence logit.

sha256 of onnx/model.onnx: 7d6e82dee82a1dca5fbddb282b3cc74571833a530de317fc22ae325c3358beeb

This conversion is used by painter to draw eyes, brows and lips on its coloring pages.

License

Apache-2.0, the same as the original model. The model and its weights are © Google LLC. This repository only changes the file format.