yolo26-hand-pose-face-detection

YOLO26 Hand Pose & Face Detection Models

Custom-trained YOLO26 models for real-time hand tracking and face detection. Built for touchless kiosk interaction via dwell-based cursor control.

Models

Both models use YOLO26's end-to-end NMS-free architecture. Output is in xyxy format — no post-processing needed.

Hand Pose Output Format

Each of the 300 detections contains 69 values:

[x1, y1, x2, y2, confidence, class_id, kp1_x, kp1_y, kp1_vis, ..., kp21_x, kp21_y, kp21_vis]
 ├─ bbox (xyxy) ─┘    │          │       └──────── 21 keypoints × 3 ────────────────────────┘
                       │          └─ always 0 (single class: hand)
                       └─ detection confidence

Face Detection Output Format

Each of the 300 detections contains 6 values:

[x1, y1, x2, y2, confidence, class_id]
 ├─ bbox (xyxy) ─┘    │          └─ always 0 (single class: face)
                       └─ detection confidence

21 Hand Keypoints

    8 (index tip) ← used for cursor
    |
    7
    |      12    16    20
    6      |     |     |
    |      11    15    19
    5      |     |     |
    |  4   10    14    18
    |  |   |     |     |
    |  3   9     13    17
    |  |   |     |     |
    |  2   |     |     |
    |  |   |     |     |
    |  1   |     |     |
    └──┴───┴─────┴─────┘
              0 (wrist)

Training Results

Hand Pose

Final metrics (epoch 100/100):

Face Detection

Final metrics (epoch 50/50):

Note: WiderFace scores appear low because the dataset includes extremely small faces (crowds, distant people). For kiosk use (single person close to camera), detection is highly reliable.

Total Training Cost

Quick Start

Download Pre-trained Models

# Clone with Git LFS
git clone https://github.com/YOUR_USERNAME/yolo26-training.git
cd yolo26-training

# Models are in models/ (tracked with Git LFS)
ls -lh models/

Export .pt to ONNX

pip install ultralytics onnx onnxslim

# Exports both FP32 and FP16 versions
bash scripts/export_onnx.sh checkpoints/yolo26_hand_pose.pt
bash scripts/export_onnx.sh checkpoints/yolo26_face.pt

Inference (Python)

from ultralytics import YOLO

# Hand pose
model = YOLO("checkpoints/yolo26_hand_pose.pt")
results = model.predict("image.jpg")

for r in results:
    keypoints = r.keypoints.xy   # [N, 21, 2] — pixel coordinates
    boxes = r.boxes.xyxy         # [N, 4] — bounding boxes
    confs = r.boxes.conf         # [N] — confidence scores

# Face detection
model = YOLO("checkpoints/yolo26_face.pt")
results = model.predict("photo.jpg")
has_face = len(results[0].boxes) > 0

Inference (C# / ONNX Runtime + DirectML)

// Load model
var session = new InferenceSession("yolo26_hand_pose.onnx", options);

// Run inference → output shape [1, 300, 69]
using var results = session.Run(inputs);
var tensor = results.First().AsTensor<float>();

// Parse: [x1, y1, x2, y2, conf, class, kp1_x, kp1_y, kp1_vis, ...]
var conf = tensor[0, i, 4];        // confidence
var indexTipX = tensor[0, i, 30];   // keypoint 8 (index tip) x
var indexTipY = tensor[0, i, 31];   // keypoint 8 (index tip) y

Train from Scratch

On RunPod (or any cloud GPU)

pip install ultralytics onnxruntime-gpu onnx onnxslim

# Hand pose (~1h45 on H200, ~3h on A40)
bash scripts/train_hand_pose.sh

# Face detection (~54min on H200, ~2h on A40)
bash scripts/train_face_detect.sh

With Docker

docker build -t yolo26-training -f docker/Dockerfile .
docker run --gpus all -v $(pwd)/models:/workspace/output yolo26-training

Recommended GPUs

Training Tips

• Hand pose: batch=512 works on 48GB+ VRAM. Lower to 128 on 24GB GPUs.
• Face detect: batch=64 recommended. WiderFace has 100+ faces/image — higher batch causes OOM.
• Early stopping: Both scripts use patience to stop early if metrics plateau.
• WiderFace download: Script auto-downloads from HuggingFace CUHK-CSE mirror (Google Drive links are unreliable).

Project Structure

yolo26-training/
├── README.md
├── .gitignore
├── .gitattributes             # Git LFS tracking for .onnx and .pt files
├── models/                    # Exported ONNX models
│   ├── yolo26_hand_pose_fp16.onnx   # FP16 (20.5 MB) — recommended
│   ├── yolo26_hand_pose_fp32.onnx   # FP32 (41.0 MB)
│   ├── yolo26_face_fp16.onnx        # FP16 (18.3 MB) — recommended
│   └── yolo26_face_fp32.onnx        # FP32 (36.4 MB)
├── checkpoints/               # PyTorch checkpoints
│   ├── yolo26_hand_pose.pt          # Hand pose (19.3 MB stripped)
│   └── yolo26_face.pt               # Face detect (19.3 MB stripped)
├── scripts/
│   ├── train_hand_pose.sh     # Hand pose training (auto-downloads dataset)
│   ├── train_face_detect.sh   # Face detection training (auto-downloads WiderFace)
│   └── export_onnx.sh         # PT → ONNX export (FP32 + FP16)
└── docker/
    └── Dockerfile             # GPU training container

Technical Details

Why YOLO26?

ONNX Compatibility

FP16 vs FP32

Datasets

License

• Training code: MIT
• Base models: Ultralytics YOLO26 — AGPL-3.0 (or Enterprise License)
• Hand Keypoints dataset: See Ultralytics docs
• WiderFace dataset: See WiderFace terms

---

Trained on April 3, 2026 using Ultralytics 8.4.33 + PyTorch 2.8.0 + CUDA 12.8 on RunPod.