LiLTInvoiceCzechV0

LiLTInvoiceCzech (V0 – Synthetic Templates Only)

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base for structured information extraction from Czech invoices.

It achieves the following results on the evaluation set:

• Loss: 0.1929
• Precision: 0.6036
• Recall: 0.7355
• F1: 0.6631
• Accuracy: 0.9645

---

Model description

LiLTInvoiceCzech (V0) is a layout-aware model based on the LiLT architecture, designed for document understanding tasks.

The model performs token-level classification with explicit use of layout information (bounding boxes), allowing it to better capture spatial relationships between invoice fields such as:

• supplier
• customer
• invoice number
• bank details
• totals
• dates

This version is trained exclusively on synthetically generated invoice templates.

---

Training data

The dataset consists of:

• synthetically generated invoices
• fixed template layouts
• associated bounding box annotations for each token

Key properties:

• consistent spatial structure
• clean and noise-free data
• precise alignment between text and layout
• no real-world documents

This represents the baseline dataset for layout-aware models in the pipeline.

---

Role in the pipeline

This model corresponds to:

V0 – Synthetic template-based dataset only

It is used to:

• establish a baseline for LiLT architecture
• compare layout-aware vs text-only models (e.g., BERT)
• evaluate the benefit of spatial features in a controlled setting

---

Intended uses

• Document AI research with layout-aware models
• Benchmarking LiLT on structured documents
• Comparison with other architectures (BERT, LayoutLMv3, etc.)
• Czech invoice information extraction

---

Limitations

• Trained only on synthetic data with fixed layouts
• Limited robustness to layout variability
• No exposure to real-world noise (OCR errors, distortions)
• Synthetic layouts may not reflect real invoice diversity

---

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 2
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.1
• num_epochs: 10
• mixed_precision_training: Native AMP

---

Training results

---

Framework versions

• Transformers 5.0.0
• PyTorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2