pelvitect-kp

pelvitect-kp — Pediatric Hip Keypoint Detection (ViTPose-Plus-Base, 4-ch)

Part of the Pelvitect framework for automated analysis of developmental dysplasia of the hip (DDH).

⚠️ Custom 4-channel input — do not load naively

This model has been fine-tuned with a 4-channel input: [R/255, G/255, B/255, seg_mask/12].

Standard transformers image processors expect 3 channels and will either error or silently produce wrong predictions. Always use the Pelvitect wrapper:

from pelvitect.analyzer import KeypointDetector

detector = KeypointDetector()   # downloads this repo automatically
kp = detector.predict(image, mask=mask)
# kp = {"tcc_r": (x, y), "tcc_l": ..., "asm_r": ..., ...}

Or use the full pipeline:

from pelvitect import Pelvitect
import numpy as np
from PIL import Image

img    = np.array(Image.open("xray.png").convert("RGB"))
result = Pelvitect(image=img).analyze()  # seg → keypoints → geometry → classification
print(result.summary())

Model description

ViTPose-Plus-Base fine-tuned on PelviSet (10.5281/zenodo.20615290). Predicts 8 anatomical keypoints (4 per hip side) from per-side ROI crops of anteroposterior pediatric pelvic radiographs.

Keypoints predicted (per side)

Inference pipeline

The model operates on per-side ROI crops, not full images:

Full image
  │
  ├─ SegFormer-B3 (pelvitect-seg)
  │      → 12-class mask
  │
  ├─ hip_roi_bbox(mask, side="r")  →  right-hip bbox
  ├─ hip_roi_bbox(mask, side="l")  →  left-hip  bbox
  │
  ├─ crop_to_vitpose(image, bbox, seg_mask)
  │      → (256×192, 4-ch)  [R, G, B, mask/12]
  │
  ├─ ViTPose forward (this model)
  │      → 4 heatmaps, 64×48
  │
  └─ kp_from_crop_space(peak_x, peak_y, scale_info)
         → (x, y) in original image coords

Training details

• Base model: usyd-community/vitpose-plus-base
• Input: 4-channel per-side hip ROI crop, 256 × 192 px
• Input ch 4: segmentation mask from pelvitect-seg, divided by 12
• Output: 4 Gaussian heatmaps per side, 64 × 48
• Dataset: PelviSet — 1,978 real-annotated cases (MTDDH DS1+DS2) + pseudo-labeled USTC-DDH and AV-DDH cases
• No horizontal flip augmentation: laterality is encoded in the mask; flip would corrupt the correspondence between image and mask channels

Citation

@dataset{pelviset2025,
  author    = {Mahdavikia, Amir M.},
  title     = {PelviSet: A Unified Pediatric Pelvic Radiograph Dataset},
  year      = {2025},
  doi       = {10.5281/zenodo.20615290},
}