mangayaku-onnx-int8

Mangayaku on-device models (int8)

Quantised ONNX models used by Mangayaku for fully on-device manga text detection and recognition. No server required.

The repository as a whole is labelled GPL-3.0, the stricter of the two licenses. Apache-2.0 is one-way compatible with GPL-3.0, so redistributing them together is fine.

What was changed

Only the manga-ocr encoder/decoder are modified: int8 dynamic quantisation of MatMul ops only. The detector is byte-identical to upstream.

Quantising Conv too would emit ConvInteger, which the ONNX Runtime CPU execution provider does not implement — the model then fails to load. The ViT patch-embedding convolution is negligible in FLOPs anyway, so nothing is lost.

Reproduce with server/tools/quantize_int8.py.

Why quantise

Measured on a Pixel 6, 4 threads, single text region:

Recognition output is character-for-character identical to fp32 (verified on both desktop and device), while the two files shrink from 461 MB to 133 MB.

Model I/O

comic-text-detector
  in  images [1,3,1024,1024] float, RGB, 0-1
  out blk    [1,64512,7]  YOLOv5 head, decoded in-graph: cx,cy,w,h,obj,cls0,cls1
      seg    [1,1,1024,1024]   (unused)
      det    [1,2,1024,1024]   (unused)

manga-ocr encoder
  in  pixel_values [1,3,224,224]  ->  out last_hidden_state [1,197,768]
manga-ocr decoder
  in  input_ids [1,L] + encoder_hidden_states [1,197,768]  ->  out logits [1,L,6144]
  start token 2, eos 3; preprocessing is greyscale, mean=std=0.5

Note manga-ocr's known behaviour: it hallucinates plausible Japanese for regions that contain no text, so only feed it crops produced by the detector.