ebim-bowl-pointer

Bowl pointer — a CPU-sized keypoint model for a mobile manipulator

Given a wrist-camera frame, returns one pixel: where the bowl is.

How it was made

Distilled from a large vision-language model. The teacher labelled every frame of 19 teleoperated grasp demonstrations; this student was trained to reproduce those answers. No hand annotation.

An earlier attempt derived labels from the arm's own kinematics and failed: the wrist camera's mounting solve carries a constant error that no gripper-offset parameter can absorb. Distillation avoids that chain entirely — the teacher and the student look at the same photograph, and neither needs to know where the arm is.

Honest limits

• Bowl only. The source episodes are bowl grasps; there are no cup or plate demonstrations to learn from.
• Recall against teacher labels is ~52%, and the misses are frames containing no bowl at all — the teacher hallucinated one on an empty table and the student declines. That is the desired behaviour, so the recall figure understates it.
• Trained on one cell. Nineteen episodes is nineteen scene layouts.

Use

from ultralytics import YOLO
m = YOLO("best.pt")
r = m.predict("frame.jpg", device="cpu", conf=0.05)[0]
u, v = r.keypoints.xy[0][0]      # the pixel

conf=0.05 is the operating point: above it every detection is trustworthy (p90 30 px); below it recall is bought with noise.