LiLTInvoiceCzechV01

LiLTInvoiceCzech (V1 – Synthetic + Random Layout)

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base for structured information extraction from Czech invoices.

It achieves the following results on the evaluation set:

• Loss: 0.1907
• Precision: 0.6326
• Recall: 0.7491
• F1: 0.6859
• Accuracy: 0.9660

---

Model description

LiLTInvoiceCzech (V1) extends the baseline layout-aware model by introducing layout variability into the training data.

The model performs token-level classification using both textual and spatial (bounding box) information to extract structured invoice fields:

• supplier
• customer
• invoice number
• bank details
• totals
• dates

Compared to V0, this version is trained on synthetically generated invoices with randomized layouts, improving robustness to spatial variations.

---

Training data

The dataset consists of:

• synthetically generated invoices based on templates
• augmented variants with randomized layout structures
• corresponding bounding box annotations

Key properties:

• variable positioning of fields
• layout perturbations (shifts, spacing, ordering)
• preserved label consistency
• fully synthetic data

This dataset introduces layout diversity, which is especially important for layout-aware models.

---

Role in the pipeline

This model corresponds to:

V1 – Synthetic templates + randomized layouts

It is used to:

• evaluate the effect of layout variability on LiLT
• compare against:
  • V0 (fixed layouts)
  • later stages with hybrid and real data (V2, V3)
• analyze how layout-aware models benefit from synthetic augmentation

---

Intended uses

• Research in layout-aware document understanding
• Evaluation of spatial robustness in NLP models
• Benchmarking LiLT against text-only models (BERT)
• Czech invoice information extraction

---

Limitations

• Still trained only on synthetic data
• Layout variability is artificial
• No real-world noise (OCR errors, distortions)
• May not fully generalize to real invoice distributions

---

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 2
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.1
• num_epochs: 10
• mixed_precision_training: Native AMP

---

Training results

---

Framework versions

• Transformers 5.0.0
• PyTorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2