documentlayoutsegmentation_YOLOv8_ondoclaynet

Model Description

This model was developed to address the challenges of Document Layout Segmentation and Document Layout Analysis by accurately segmenting a document page into its core components. These components include the title, captions, footnotes, formulas, list items, page footers, page headers, and pictures. The motivation behind creating this model stems from the need to enhance the understanding and accessibility of document content, facilitating a wide range of applications such as automated content extraction, document summarization, and improved accessibility features. By providing precise segmentation of these elements, the model aims to support various downstream tasks that rely on the structural understanding of document layouts, enabling more efficient and effective processing and analysis of document content.

Example Output

Training Data:

• Source: DocLayNet, IBM (https://github.com/DS4SD/DocLayNet)
• Classes: 11 classes (Caption, Footnote, Formula, List-item, Page-footer, Page-header, Picture, Section-header, Table, Text, and Title)
• Pages: 80,863 document pages

Performance

Metrics:

• Precision: 0.98
• Recall: 0.97
• F1: 0.97
• mAP50: 0.99
• mAP50-95: 0.95

Confusion Matrix:

Usage

Example Code

To use the model, follow this example code:

from ultralytics import YOLO
from PIL import Image, ImageDraw
import pathlib

# List of sample images to process
img_list = ['sample1.png', 'sample2.png', 'sample3.png']

# Load the document segmentation model
docseg_model = YOLO('yolov8x-doclaynet-epoch64-imgsz640-initiallr1e-4-finallr1e-5.pt')

# Process the images with the model
results = docseg_model(source=img_list, save=True, show_labels=True, show_conf=True, show_boxes=True)

# Initialize a dictionary to store results
mydict = {}

# Extract and store the paths and coordinates of detected components
for entry in results:
    thepath = pathlib.Path(entry.path)
    thecoords = entry.boxes.xyxy.numpy()
    mydict.update({thepath: thecoords})

Model Details

• Model Name: DILHTWD/documentlayoutsegmentation_YOLOv8_ondoclaynet
• Publisher: Data Intelligence Lab, Hochschule für Technik und Wirtschaft Dresdem
• Model Version: 1.0.0
• Model Date: 2024-03-17
• License: AGPL-3.0
• Architecture: YOLOv8 XL (https://github.com/ultralytics/ultralytics)
• Task: Document Layout Segmentation, Document Layout Analysis