anglenet

AngleNet

AngleNet predicts the reading orientation and rotation angle (0° to 180°) for manga text boxes. It is a lightweight post-processing model (+0.22 ms per box / 1.0 MB) for meiki.text.detect.v0. It converts straight boxes into oriented polygons and keeps the 95.66% detector precision.

Native meiki recognition has high error on vertical manga text (~23% end-to-end CER). Adding AngleNet directly to meiki recognition does not fix this limit. This pipeline uses PP-OCRv6 Small Rec instead. Unrotated boxes still fail on tilted text (58.4% CER). AngleNet fixes this problem. It reduces median slant error from 9.15° to 2.87° and lowers tilted dialogue CER from 58.4% to 11.4%. The full pipeline reaches 12.8% end-to-end CER (8.9% matched CER, 86.54% Tilt IoU >= 0.5).

---

Full pipeline (detection + recognition)

Evaluated on 300 unseen manga and webtoon test pages (2,631 dialogue lines) with a shared PP-OCRv6 Small recognizer (23.0 MB FP16) to isolate the exact impact of bounding box detection and reading orientation on recognition accuracy.

Bold = Best, <u>Underline</u> = 2nd place.

Angle detection

Bold = Best, <u>Underline</u> = 2nd place. Sorted by Tilt IoU >= 0.5.

* Measured per page on strictly 1 single CPU thread (single-core edge/mobile/WASM baseline, ONNX Runtime).

---

Architecture

• Student model (v0.1 base, v0.1 distill):
  • 4-stage Re-parameterizable Convolutional Network (RepBlock with 3x3 and 1x1 training branches fused into single 3x3 convolutions at deployment).
  • Multi-aspect pooling head combining Global Average Pool (1x1), Vertical Column Pool (3x1), and Horizontal Line Pool (1x3).
  • Two output heads: 180-bin Circular Smooth Label (CSL) angle classifier and a binary tilt classifier.
  • Total parameters: 516,405 (2.00 MB FP32 / 1.02 MB FP16).
  • 1-Thread CPU latency per crop: 0.216 ms (64x64) / 0.438 ms (96x96).
• Teacher model (v0.1 pro):
  • 5-stage RepBlock network with Coordinate Attention (CoordAtt) at 128x128 resolution.
  • Total parameters: 3,842,105 (14.45 MB FP32 / 7.26 MB FP16).
  • 1-Thread CPU latency per crop: 2.976 ms (128x128).

---

Datasets

The model was trained on 345,265 samples:

• Manga and webtoon (Manga109, AnimeText, others): 261,543 samples
• Synthetic manga crops: 35,000 samples
• Japanese scene text (JA-WildText): 48,722 samples

---

Training setup

• Dataset size: 310,737 train / 34,528 val
• Batch size: 512
• Steps per epoch: 607
• Epochs: 15 to 20
• Total steps: 9,105 to 12,140
• Optimizer: AdamW (lr = 0.002, weight decay = 0.0001)
• Schedule: Cosine Annealing
• Distillation: Student is trained via ground-truth CSL cross-entropy and KL-divergence distillation from the 128x128 Teacher (temperature = 2.0).

---

Quick start: inference

import cv2, math, numpy as np, onnxruntime as ort
from huggingface_hub import hf_hub_download

# Load models (auto-download from Hugging Face)
meiki_path = hf_hub_download(repo_id="rtr46/meiki.text.detect.v0", filename="meiki.text.detect.small.v0.onnx")
angle_path = hf_hub_download(repo_id="Kellenok/anglenet", filename="anglenet_v0_1_distill_64x64.onnx")

meiki_sess = ort.InferenceSession(meiki_path)
angle_sess = ort.InferenceSession(angle_path)

def letterbox_crop(crop_gray, target_size=64):
    h, w = crop_gray.shape[:2]
    scale = min(target_size / max(1, w), target_size / max(1, h))
    nw, nh = max(1, int(w * scale)), max(1, int(h * scale))
    resized = cv2.resize(crop_gray, (nw, nh), interpolation=cv2.INTER_AREA)
    pad = np.zeros((target_size, target_size), dtype=np.uint8)
    pad[(target_size - nh) // 2 : (target_size - nh) // 2 + nh,
        (target_size - nw) // 2 : (target_size - nw) // 2 + nw] = resized
    return (pad.astype(np.float32) / 255.0)[None, None, :, :]

def get_rotated_polygon(x1, y1, x2, y2, rot_deg, width_scale=1.05):
    cx, cy = (x1 + x2) / 2.0, (y1 + y2) / 2.0
    w, h = (x2 - x1) * width_scale, (y2 - y1)
    rad = math.radians(rot_deg)
    cos_a, sin_a = math.cos(rad), math.sin(rad)
    hw, hh = w / 2.0, h / 2.0
    corners = [(-hw, -hh), (hw, -hh), (hw, hh), (-hw, hh)]
    return [[round(cx + dx*cos_a - dy*sin_a, 1), round(cy + dx*sin_a + dy*cos_a, 1)] for dx, dy in corners]

def detect_and_orient(image_bgr, conf_thresh=0.44):
    H, W = image_bgr.shape[:2]
    img_gray = cv2.cvtColor(image_bgr, cv2.COLOR_BGR2GRAY)
    
    # 1. Run Meiki Detector (640x640)
    img_640 = cv2.resize(image_bgr, (640, 640)).transpose(2, 0, 1)[None].astype(np.float32) / 255.0
    orig_sz = np.array([[H, W]], dtype=np.int64)
    _, boxes, scores = meiki_sess.run(None, {"images": img_640, "orig_target_sizes": orig_sz})
    
    # 2. Filter boxes by confidence
    valid = scores[0] >= conf_thresh
    boxes = boxes[0][valid]
    
    # 3. Predict angles with AngleNet
    polygons = []
    for (x1, y1, x2, y2) in boxes:
        w, h = x2 - x1, y2 - y1
        if w < 4 or h < 4:
            continue
            
        # 10% context margin around crop
        pw, ph = int(w * 0.10), int(h * 0.10)
        crop = img_gray[max(0, int(y1 - ph)):min(H, int(y2 + ph)), max(0, int(x1 - pw)):min(W, int(x2 + pw))]
        
        # Infer AngleNet (0.11 ms on CPU)
        input_tensor = letterbox_crop(crop, target_size=64)
        csl_logits, tilt_logit = angle_sess.run(None, {"input": input_tensor})
        
        # CSL continuous trigonometric decoding
        probs = np.exp(csl_logits[0] - np.max(csl_logits[0]))
        probs /= np.sum(probs)
        bins = np.arange(180, dtype=np.float32)
        pred_deg = (0.5 * np.degrees(np.arctan2(
            np.sum(probs * np.sin(np.radians(2.0 * bins))),
            np.sum(probs * np.cos(np.radians(2.0 * bins)))
        ))) % 180.0
        
        # Compute rotation delta
        is_vert = (h >= w)
        rot_delta = (pred_deg - 90.0) if is_vert else (pred_deg if pred_deg <= 90.0 else pred_deg - 180.0)
        is_tilted = (abs(rot_delta) >= 0.5) and ((max(w, h) / max(1.0, min(w, h))) >= 1.1)
        
        poly = get_rotated_polygon(x1, y1, x2, y2, rot_delta, width_scale=1.05) if is_tilted else [[x1, y1], [x2, y1], [x2, y2], [x1, y2]]
        polygons.append(poly)
        
    return polygons