layoutlmv3-cord-extraction

LayoutLMv3 fine-tuned on CORD-v2 for receipt field extraction

Token classification over receipt images, extracting 59 BIO-tagged field types (menu items, quantities, unit prices, subtotals, tax, service charges, payment amounts).

Part of a document-intelligence pipeline: Perlious-Savage/financial-doc-intelligence.

Results

Evaluated on the CORD-v2 test split (100 documents, 1,309 entities).

Scoring is entity-level: a prediction counts only when its type and its full span match the reference exactly. Partial extractions score zero, because a partially extracted field is not a usable one. Implementation: src/metrics.py.

Where the gain comes from

The rule baseline handles fields printed next to their own label and scores exactly zero on fields identified by position alone. Those are the fields this model exists for:

Known weaknesses

Every remaining weak field is a rare class: menu.etc (3 test examples, F1 0.000), total.total_etc (3, 0.000), menu.itemsubtotal (6, 0.000), total.emoneyprice (2, 0.400), menu.discountprice (10, 0.571). Two are catch-all etc categories, where scarcity compounds with genuine ambiguity about what belongs in them. This is a data-coverage limitation, not an architectural one.

Training

Validation F1 by epoch: 0.820, 0.923, 0.921, 0.950, 0.946, 0.952, 0.955, 0.955 — plateaus from epoch 6, so the final two epochs added nothing.

Limitations

• Single training run. No repeated seeds, no confidence intervals. With 100 test documents the interval around 0.948 is not tight; a rerun would move the third decimal.
• Receipts, not KYC. CORD is Indonesian retail receipts. This measures document extraction, not regulatory or compliance performance.
• Indonesian number formatting. Amounts use . as a thousands separator (12.000 is twelve thousand). Parsing logic assumes this convention.
• Comparable published work (nielsr/layoutlmv3-finetuned-cord) reports 0.964 F1 under a different evaluation setup — context, not a like-for-like comparison.

Usage

from transformers import AutoProcessor, LayoutLMv3ForTokenClassification

processor = AutoProcessor.from_pretrained("perlious-Savage/layoutlmv3-cord-extraction", apply_ocr=True)
model = LayoutLMv3ForTokenClassification.from_pretrained("perlious-Savage/layoutlmv3-cord-extraction")

License

Inherits CC-BY-NC-4.0 from microsoft/layoutlmv3-base (non-commercial). Training data: CORD-v2, CC-BY-4.0.