form-classifier-layoutlmv3

Form Classifier LayoutLMv3

This is a fine-tuned LayoutLMv3ForSequenceClassification checkpoint for classifying a single document page as Form or Non-Form. It combines the page image with OCR words and normalized two-dimensional bounding boxes.

This model requires an external OCR/layout step. It is not a plain image-classification model and is not expected to work through the standard Hugging Face image-classification widget.

Model details

Class definitions

• Form: a structured page intended to collect, communicate, or organize information through labelled fields, boxes, selections, or a comparable form-like layout.
• Non-Form: a page without that structure, such as free-form prose or other document content outside the project's form-routing definition.

These definitions are project-specific. Users should validate borderline document types against their own routing policy.

Intended use

The checkpoint is intended for research and non-commercial document-routing experiments involving English-language, single-page scanned documents. It can be used to decide which processing pipeline should inspect a page next.

It is not intended to make legal, compliance, identity-verification, credit, employment, medical, or other high-impact decisions. A predicted label should not be treated as evidence that a document is genuine, complete, or safe.

Input contract

For each page, callers must provide:

1. an RGB image;
2. OCR words in reading order; and
3. one [x0, y0, x1, y1] bounding box per word, normalized to the LayoutLM coordinate range of 0 to 1000.

The saved processor has apply_ocr=false. The companion Form Classifier code uses Docling for OCR and layout extraction. Passing an image without matching words and boxes does not reproduce the evaluated pipeline.

Transformers example

The following example demonstrates the model interface after OCR has already produced words and boxes:

from PIL import Image
import torch
from transformers import (
    LayoutLMv3ForSequenceClassification,
    LayoutLMv3Processor,
)

model_id = "AIwithSakthivel/form-classifier-layoutlmv3"

processor = LayoutLMv3Processor.from_pretrained(model_id, apply_ocr=False)
model = LayoutLMv3ForSequenceClassification.from_pretrained(model_id)
model.eval()

image = Image.open("document.png").convert("RGB")

# Replace these values with output from an OCR/layout system. Boxes must be
# normalized to 0..1000 and aligned one-to-one with the words.
words = ["First", "name"]
boxes = [[90, 120, 180, 155], [200, 120, 330, 155]]

inputs = processor(
    image,
    text=words,
    boxes=boxes,
    padding="max_length",
    truncation=True,
    max_length=512,
    return_tensors="pt",
)

with torch.no_grad():
    probabilities = model(**inputs).logits.softmax(dim=-1)[0]

class_id = int(probabilities.argmax())
print(
    {
        "label": model.config.id2label[class_id],
        "confidence": float(probabilities[class_id]),
    }
)

Confidence values are uncalibrated softmax probabilities and should not be interpreted as guaranteed correctness.

Evaluation

Project evaluation records report the complete saved pipeline on a 200-page test collection spanning six sources. The test set contained 120 Form pages and 80 Non-Form pages. The result artifact did not embed a model-weight checksum, so the association with this delivered checkpoint is based on the project release structure rather than immutable experiment lineage.

Evaluation composition

The all-positive CORD, KYC, and passport subsets measure recall on those sources, not specificity. The collection is relatively small, source-dependent, and should not be treated as proof of performance on unrelated production data.

Project records also contain a corrected, balanced RVL-derived evaluation with 999 valid predictions: 92.99% accuracy, 92.99% balanced accuracy, and 92.93% Form F1. Its filename/label reconstruction was fragile and its result artifact did not embed a weight checksum, so it is reported as a stress test rather than the primary release result.

Training and data provenance

Project materials describe a fine-tuning corpus of approximately 900 pages assembled from document-classification sources and project-specific examples. The delivered checkpoint does not include immutable training/evaluation IDs or a complete machine-readable training log. Conflicting historical records exist for the named backbone and hyperparameters, so exact optimizer settings, split, seed, and validation metrics are intentionally not asserted here.

The architecture, tensor dimensions, label count, processor configuration, SafeTensors integrity, and evaluation tables above were inspected from the delivered artifacts and saved result files. Dataset files are not distributed with this repository and remain subject to their original licenses and terms.

Limitations

• Performance is only documented for English-language pages.
• OCR quality and bounding-box accuracy directly affect predictions.
• Handwriting, rotated or distorted scans, and unusual layouts were not comprehensively evaluated.
• Long pages are truncated to the first 512 processed tokens.
• Source-specific class imbalance can make accuracy misleading.
• The model has not been calibrated; high confidence can still be wrong.
• Training lineage is incomplete, so the exact fine-tuning run is not fully reproducible from the published artifacts alone.
• Distribution shift may require new evaluation or fine-tuning.

Bias, privacy, and safety

Document collections can contain personal or sensitive information. Do not upload private documents to third-party systems without an appropriate legal basis and data-handling policy. Review errors by source, scan quality, language, and document type before deployment. Maintain a human-review path for uncertain or consequential routing decisions.

License

The model is a derivative of microsoft/layoutlmv3-base and is released under CC BY-NC-SA 4.0. This license restricts use to non-commercial purposes and requires attribution and share-alike distribution. See LICENSE and the base model card. Dataset licenses are separate.

Citation

@inproceedings{huang2022layoutlmv3,
  title={LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking},
  author={Huang, Yupan and Lv, Tengchao and Cui, Lei and Lu, Yutong and Wei, Furu},
  booktitle={Proceedings of the 30th ACM International Conference on Multimedia},
  year={2022}
}

Checkpoint integrity

Checksums for every published artifact are provided in checksums.sha256.