layoutLm-v3-invoice-automation

Model Card for Model ID

The model is used for invoice automation process.It predicts key and corresponding value from a given invoice.

This modelcard aims to be a base template for new models. It has been generated using

Model Details

Layoutlm is fine tuned for the invoice processing .The reason why i choose layoutlm v3 is beacause, NLP ,NER and layout details will increase the accuracy of the output.Layoutlm v3 provides this feature and much apt for our use case scenario. The NER tagging also helps a lot.

Model Description

Layoutlm v3 for invoice automation

• Developed by: Mohammed Ashid
• Finetuned from model [optional]: Layout lm v3

Uses

Invoice Processing

Direct Use

I got a model which has around 70% accuracy with just 10 data. confidence =.91.More fine tuning or training can generate more accuracy. Since the use case is for invoices,accuracy should be greater than 90%

Bias, Risks, and Limitations

The quality of ocr data affects this.

Recommendations

Different OCR technologies are available.i find pdf plumber better for digital pdf.or else we can use paddleOcr or tesseract

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Label studio is used to create labelling.Either we can choose key point labelling or optical character recognition. The code may slightly change according to the labelling template used. I have used key point labelling.then i exported the label to json format.i have used python code for creating the data setafter that. The images used for labelling will autmatically be saved in the root directory by label studio itself ,take those images and save it in a folder named layoutlmv3. The labelled json file along with corresponding images used for labelling is provided to the dataCreation.py code which will perform OCR and fetch the corresponding data and bounding boxes needed and it also cretes a testing and training file. save the testing and training file in the layoutlmv3 folder. save all the above code and layoutlmv3 in a single folder. make sure that the layoutlm.py file propperly maps the path of layoutlmv3 folder. Run loaddata.py which will generate atest folder in which your fine tuned model is created.Different checkpoints can be seen.