receipt-detection

View on Pictograph · ClearObject · Object Detection · YOLOX-M · FP32 · Apache 2.0

Overview

This is an object detection model built on YOLOX-M. It expects 640x640 input and runs in FP32 precision. The ONNX weights are in this repo; run them with the Pictograph SDK (below) or open the model on Pictograph to test it in-browser, call the hosted API, or deploy it as an always-on endpoint.

Performance

Headline: 87.2% mAP.

Classes

Class index matches the model's output order.

Training

Use this model

The weights ship as ONNX (yolox-c4b9f885.onnx) with a config.json. The Pictograph SDK applies the model's exact pre/post-processing for you:

Pictograph SDK

pip install "pictograph[inference]"

from huggingface_hub import hf_hub_download
from pictograph import load_model, DetectionModel

onnx = hf_hub_download("pictograph/receipt-detection", "yolox-c4b9f885.onnx")
config = hf_hub_download("pictograph/receipt-detection", "config.json")

model: DetectionModel = load_model(onnx, config, task="object_detection")
result = model.predict("photo.jpg")   # applies the model's pre/post-processing
print(result)

<details> <summary><b>Raw ONNX Runtime</b> (no SDK)</summary>

from huggingface_hub import hf_hub_download
import onnxruntime as ort, numpy as np
from PIL import Image

onnx = hf_hub_download("pictograph/receipt-detection", "yolox-c4b9f885.onnx")
sess = ort.InferenceSession(onnx, providers=["CPUExecutionProvider"])
S = 640                                              # model input size

# letterbox to SxS, pad 114, CHW float32 (YOLOX uses raw 0-255, NO /255)
img = Image.open("photo.jpg").convert("RGB")
w0, h0 = img.size; r = min(S / h0, S / w0)
resized = img.resize((round(w0 * r), round(h0 * r)))
canvas = np.full((S, S, 3), 114, np.uint8)
canvas[: resized.height, : resized.width] = np.array(resized)
x = canvas.transpose(2, 0, 1)[None].astype(np.float32)

pred = sess.run(None, {sess.get_inputs()[0].name: x})[0][0]   # [8400, 5+C]

# decode the YOLOX grid (strides 8/16/32)
grids, strides = [], []
for s in (8, 16, 32):
    n = S // s
    xv, yv = np.meshgrid(np.arange(n), np.arange(n))
    grids.append(np.stack((xv, yv), 2).reshape(-1, 2))
    strides.append(np.full((n * n, 1), s))
grids = np.concatenate(grids); strides = np.concatenate(strides)
pred[:, :2] = (pred[:, :2] + grids) * strides
pred[:, 2:4] = np.exp(pred[:, 2:4]) * strides

xy, wh = pred[:, :2], pred[:, 2:4]
boxes = np.concatenate([xy - wh / 2, xy + wh / 2], 1) / r   # xyxy, source px
s = pred[:, 4:5] * pred[:, 5:]
cls, conf = s.argmax(1), s.max(1)
keep = conf > 0.3
boxes, conf, cls = boxes[keep], conf[keep], cls[keep]

def nms(b, sc, iou=0.45):
    order = sc.argsort()[::-1]; out = []
    while len(order):
        i = order[0]; out.append(i)
        x1 = np.maximum(b[i, 0], b[order[1:], 0]); y1 = np.maximum(b[i, 1], b[order[1:], 1])
        x2 = np.minimum(b[i, 2], b[order[1:], 2]); y2 = np.minimum(b[i, 3], b[order[1:], 3])
        inter = np.clip(x2 - x1, 0, None) * np.clip(y2 - y1, 0, None)
        ai = (b[i, 2] - b[i, 0]) * (b[i, 3] - b[i, 1])
        ar = (b[order[1:], 2] - b[order[1:], 0]) * (b[order[1:], 3] - b[order[1:], 1])
        order = order[1:][inter / (ai + ar - inter + 1e-9) < iou]
    return out

final = nms(boxes, conf)
print(boxes[final], conf[final], cls[final])   # [x1,y1,x2,y2], score, class index

</details>

• Test it in-browser, call the hosted API, or deploy an endpoint: open the model on Pictograph.
• SDK + local inference guide: docs.

License

Released under Apache 2.0.

Citation

This model uses the YOLOX-M architecture:

@article{ge2021yolox,
  title={YOLOX: Exceeding YOLO Series in 2021},
  author={Ge, Zheng and Liu, Songtao and Wang, Feng and Li, Zeming and Sun, Jian},
  journal={arXiv preprint arXiv:2107.08430},
  year={2021}
}

---

Trained and published with Pictograph - annotate, train, and deploy from one API.