pi-r2

pi-r2 — DETR for document layout detection (loc_beyond_words)

Fine-tuned DETR (ResNet-50) for detecting layout regions in historical newspaper page scans, trained on biglam/loc_beyond_words (2,846 train / 712 validation images, 7 classes).

Model / license

• Base model: facebook/detr-resnet-50 — Apache-2.0 (open license, free to share).
• The fine-tuned model inherits the Apache-2.0 license.
• Dataset biglam/loc_beyond_words is CC0-1.0 (public domain).

Classes (7)

Photograph, Illustration, Map, Comics/Cartoon, Editorial Cartoon, Headline, Advertisement

Validation results (COCO-style metrics on the held-out validation split, 712 images)

Per-class AP

Note: AP_small/AP_medium/AP_large are computed on the resized evaluation inputs, not original pixel areas; treat small-object numbers cautiously.

Training details

• Recipe: standard DETR fine-tuning (AdamW, lr=1e-4, backbone lr=1e-5, wd=1e-4, grad clip 0.1, fp16, linear warmup/decay, batch size 2 with grad accumulation 2).
• Input: images resized to shortest edge 480px / longest edge 800px (aspect kept), RGB.
• Epochs: up to 15 with early stopping (patience 3 on [email protected]) and validation every 4 epochs.
• Hardware: 1x NVIDIA T4 (16GB) via Hugging Face Jobs (t4-small).
• Cost: < $5 total GPU compute.

Quickstart

from transformers import DetrImageProcessor, DetrForObjectDetection
from PIL import Image
import torch

processor = DetrImageProcessor.from_pretrained("harness-race/pi-r2")
model = DetrForObjectDetection.from_pretrained("harness-race/pi-r2")

image = Image.open("page.png").convert("RGB")
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

results = processor.post_process_object_detection(
    outputs, threshold=0.5, target_sizes=[(image.height, image.width)])[0]
for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
    print(model.config.id2label[label.item()], round(score.item(), 3), [round(v, 1) for v in box.tolist()])

Files

• model.safetensors, config.json, preprocessor_config.json — fine-tuned model + processor
• val_metrics.json — full COCO validation metrics (incl. per-class AP)
• figures/ — sample predictions on the validation set
• train_detr.py, requirements.txt — training code for reproducibility

Limitations

• Trained on grayscale newspaper scans; color documents may perform worse.
• Classes are heavily imbalanced (Headline/Advertisement dominate; Map/Editorial Cartoon are rare).
• Small text-heavy regions may be missed at 480–800px input resolution.