yolo11-doc-layout-research-papers

YOLOv11 Document Layout — Research Papers

A collection of YOLOv11 document-layout detectors fine-tuned on manually annotated pages from real arXiv research papers. They locate the twelve region types below on a page image, which is what makes layout-aware chunking of a paper possible: text can be split on the paragraph and section boundaries the model found, instead of on a fixed character count that cuts through the middle of a table.

Training code: mudit-choudhary/yolo11-doc-layout-research-papers. The full pipeline is there: PDF rendering, dataset splitting, fine-tuning, evaluation, latency benchmarking, and this card's own generator. It is tested and runs end to end. This repository holds weights and metrics only; the training images are not included, for the reasons given under Limitations.

Every variant lives in its own subfolder of this one repository, numbered oldest to newest. Each has its own README.md with its full training configuration.

Subfolder names read NN-yolo11<size>-<training resolution>, so 12-yolo11s-1024 is the twelfth model trained, a YOLO11-small at 1024 pixels. The number is chronological, so the folder listing shows the order the collection was built in.

An augexp suffix marks the failed augmentation experiment described below. It is there because a higher number otherwise reads as newer and better, and these are newer and worse.

Anything else you need in order to choose is in the Notes column of the table below, and in each variant's own card, rather than encoded in a folder name.

Fine-tuned from Armaggheddon/yolo11-document-layout.

Which one to use

Two answers, depending on what you are doing.

12-yolo11s-1024 for accuracy on the classes that matter. It has a clean single-pass lineage and leads on Table (0.985) and Footnote, which decide where a chunk boundary falls. It costs 37 ms per page.

11-yolo11n-1024 for throughput. It scores 0.766 against 0.769, a difference well inside noise, at 19 ms per page. Half the latency for three thousandths of mAP is the better engineering trade for most pipelines, and it is the choice to default to if you are processing pages in bulk.

If pages are small or the corpus is huge, 05-yolo11n-640 runs at 9 ms and still reaches 0.738, though small classes such as Footnote and Page-header suffer at 640.

08-yolo11n-1024 scores about 0.002 higher than either, which is also within noise. It is not recommended because it is the product of three successive fine-tuning passes over datasets that no longer exist in their original form, so it cannot be reproduced. Prefer a lineage you can rebuild.

Reading the table

Passes is how many successive fine-tuning runs produced the checkpoint, each starting from the previous one's output.

1 means it was fine-tuned once from a published base model, so you can reproduce it with a single command. 2 or more means the earliest passes used datasets that no longer exist in their original form, so it cannot be rebuilt from scratch, and it has been shaped by several dataset versions rather than one.

ms/page is median inference latency at batch 1 on a GTX 1650, each model at its own resolution, excluding image decode. Add roughly 60 to 80 ms per page for decode to get a wall-clock figure.

Precision, recall and mAP50 are on each variant's own card, alongside its full training configuration.

Pass count says nothing about accuracy. The early datasets overlap today's held-out papers by up to 19%, so the scores were checked for inflation and none was found: single-pass models show the same gap between overlapping and non-overlapping papers, meaning those pages are simply easier for everything. The numbers below are comparable across every row.

Variants whose name ends -augexp are deliberately published failures. They carry an augmentation bundle that regressed every model it was applied to, by 0.016 to 0.060 mAP50-95. They are here so the result is reproducible, not because they are worth deploying. That sweep changed seven settings at once, so it identifies a harmful combination rather than a single culprit; copy_paste at copy_paste_mode="flip" is the prime suspect, since it mirrors pasted crops even when whole-image flipping is disabled.

Variants

Measured on the held-out val split unless marked otherwise, each at the image size it was trained at. mAP50-95 is the number worth reading: mAP50 counts a box as correct when it overlaps the ground truth by half, which flatters a model that finds a region without bounding it tightly, and tight bounds are exactly what a chunking consumer needs.

Where the recommended model is strong and weak

The headline number is an average over twelve classes that range from 0.985 to 0.360, so it hides more than it shows. Table and Text, the two classes that most affect where a chunk boundary falls, are the strongest. Page-footer is the one real weakness, and it is a labelling problem rather than a model one: see Limitations.

Speed against accuracy

Latency is decided almost entirely by architecture and resolution, so the variants fall into three tight bands. Within the 1024 band, nine yolo11n variants span 0.01 mAP at identical speed.

Precision against recall

A wide gap means the model leans one way. High precision with low recall misses regions; the reverse invents them. The early runs are visibly unbalanced and the later ones are not, which is most of what fine-tuning bought.

Usage

from huggingface_hub import hf_hub_download
from ultralytics import YOLO

weights = hf_hub_download(
    repo_id="darkdwine/yolo11-doc-layout-research-papers",
    filename="12-yolo11s-1024/best.pt",
)
model = YOLO(weights)

results = model.predict("page.jpg", imgsz=1024, conf=0.2)
for box in results[0].boxes:
    class_name = results[0].names[int(box.cls)]
    x1, y1, x2, y2 = box.xyxy[0].tolist()
    print(class_name, round(float(box.conf), 3), [round(v) for v in (x1, y1, x2, y2)])

Every variant also publishes best.onnx, the same weights with a dynamic batch and page size, for runtimes without PyTorch. Non-maximum suppression is not part of the graph.

Swap the subfolder in filename to load a different variant. Run inference at the image size that variant was trained at; smaller inputs cost small classes such as Footnote and Page-header first, since those regions are only a few pixels tall once a page is scaled down.

Classes

The first eleven classes are inherited from the DocLayNet-style base model. Authors is a twelfth class added for research-paper front matter, which the base taxonomy has no equivalent for.

Training data

Pages rendered from arXiv PDFs at 300 DPI and annotated by hand in X-AnyLabeling, exported in YOLO horizontal-bounding-box format.

850 pages from 566 papers are annotated and used for training and evaluation. They are a labelled subset of a much larger unlabelled pool of roughly 22,000 rendered pages; the rest of that pool exists so later annotation rounds have material to draw on, and was not used to train these models.

The dataset is split by paper rather than by page: every page of a given paper lands in the same split, so a model is never validated on a layout it saw during training. Pages of one paper share a template, so a page-level split would leak layout across the boundary and inflate the scores.

Limitations

• Page-footer boxes are loose. The class is found reliably but bounded poorly, at roughly 0.36 mAP50-95 against 0.77 overall, consistently across every model size and resolution tested. The pattern points at inconsistent annotation tightness rather than model capacity.
• Research papers only. Training pages are arXiv preprints, mostly two-column and in English. Invoices, forms, slides and handwriting are out of distribution.
• Rendered pages, not photographs. Training images come from digital PDF rendering, so skew, shadow and camera perspective are unrepresented.
• Reading order is not predicted. The models output regions, not their sequence. Ordering is left to the consumer.

License and attribution

Released under AGPL-3.0, inherited from Ultralytics YOLO, which these models were trained with. Note the AGPL network clause: serving these weights to users over a network obliges you to offer them the corresponding source. Ultralytics sells an Enterprise License for use without that obligation.

Fine-tuned from Armaggheddon/yolo11-document-layout, which is MIT-licensed and was itself trained on DocLayNet (CDLA-Permissive-1.0).

Annotations were made with X-AnyLabeling (GPL-3.0). That licence covers the tool, not the labels produced with it.

Training images are not distributed

These repositories contain weights, metrics and plots only. The annotated page images are not published, because they are rendered from arXiv preprints whose licences vary per paper and often do not permit redistribution. Ultralytics' train_batch and val_batch debugging mosaics are excluded for the same reason.