comic-text-mask

comic-text-mask

Binary text-mask segmenter for the comic-localizer cleaning pipeline. It runs on a detector's text-region crop and returns a per-pixel "text vs not-text" mask that feeds LaMa inpainting. Locating and grouping text is the detector's job, not this model's.

Training data (see the training repo): synthetic text rendered onto (a) procedurally generated flat surfaces with synthetic clutter and (b) real cleaned comic pages, framed as detector-style crops. Text is Latin, Japanese (kana + kanji, horizontal and vertical), Korean, and Chinese, with a large fraction of random-glyph runs so rare characters are covered. No source imagery is redistributed.

Validation (held-out synthetic + real crops): IoU 0.906, precision 0.949, recall 0.952.

Use

import json, torch
from huggingface_hub import hf_hub_download

meta = json.load(open(hf_hub_download("TareHimself/comic-text-mask", "tm_meta.json")))
model = torch.jit.load(hf_hub_download("TareHimself/comic-text-mask", "model.pt")).eval()
S = meta["imgsz"]

# letterbox `rgb` (H,W,3 uint8) into an SxS square, pad 0, keep the paste box
# ... then:
x = torch.from_numpy(square).permute(2, 0, 1).unsqueeze(0)      # (1,3,S,S) uint8
prob = model(x)[0, 0].numpy()                                   # (S,S) float
mask = (prob > meta["threshold"]).astype("uint8") * 255
# crop the paste box back out and resize to the original size

Or load the raw weights with segmentation-models-pytorch:

import segmentation_models_pytorch as smp
model = smp.from_pretrained("TareHimself/comic-text-mask")   # normalisation NOT baked in

model.pt has /255, ImageNet normalisation, and the final sigmoid baked into the graph; it expects letterboxed uint8 RGB. model.safetensors is the pristine network.