Pipeline Train run does not improve ML model

Feedback AI Center
, , , ,
1

Hello,
I have a simple problem with one of the extracted row fields for one of the invoices of the template im training to generate a model for. The column value is sometimes just missing for every other row and I retried training the model through a pipeline manually with those 5-6 documents that have errors. I have added them through the Data Labeling and labelled them accordingly but after the train run, the same problem occured for those same 5-6 invoices. Any ideas what could cause this bug/issue?

2

Hi @DannyVelkov ,

Could you let us know some more details of the dataset size used for the initial training and how much of it had the type of data/template that had errors as you have mentioned ?

Also, for the next Training/Re-training have you Selected the initial Package version or the latest trained one ?

3

Well the dataset is around 500 mb and the invoices which are a problem are currently around 50.
I trained the latest version two times. Still the same results

4

from my understanding you’ve only trained 50 invoices? What’s the number of fields you are extracting?

You can go to your document manager and check if you have labelled sufficient data for all your fields, if you have labelled correctly, the indicators should be green, not red as seen below.

Note -This is similar if you’re using invoices model, your number of invoices to be trained will be a smaller value, but you should still make sure it’s in the green range.

image
image878×562 26.6 KB

Next, you should run an evaluation pipeline to check if all your fields have trained and extracted properly. Once you do these steps, it will be a clear indicator to know if your model is properly trained or not.

1 Like
5

They are the same fields (around 9 and a table), spanning different layouts for the templates.

6

how many different layouts do you have? You might have to train around 3-5 samples per layout.

Also please run an evaluation pipeline. the confidence scores and metrices will really help you understand your model performance.
Your evaluation dataset should be a small size around 20% of the size of your train dataset, but it should have all variations and should be unseen samples (ones you have not labelled in training)

7

There are 3 layouts. The newest one is the problem. How can I select a percentage of the invoices to do an evaluation run?

8

@DannyVelkov ,

Here, I want to know what kind of problem you are getting? whether your pipeline failed or you are not getting proper accuracy?

Thanks