On-prem document understanding Confidence scores always 100%

Help Document Understanding
1

Working on an on-prem AI center and automation suite installation.
We trained a document understanding model with relatively small (50 documents) sample size.
Unfortunately, ml model always returns its predictions with 100% confidence.
Note that this is not a validated document (not verified through action center nor validation station) and has never been in training dataset.

I tested over 20+ different documents.

Current dependencies:

image
image449×188 5.52 KB

Data extraction scope properties:

image
image679×599 12.4 KB

An example of the confidence levels:

image
image324×163 1.53 KB

Note that my pdf file is text based so sometimes i get 100% ocr confidence, sometimes not -which is expected-.

2

@E_Metin

confidence is fine but is the data being extracted correctly?

also sometimes because of excel formatting you might see as 1 can you check by selecting the cell and checking in the top display or check the value from locals pane in studio when in debug mode

cheers

1 Like
3

@E_Metin

Most probably it’s excel rounding off the confidence score above 0.5 to 1 that’s 100%

Check the score in debug mode as well before writing it to the excel file.

4

Thanks but this is not an excel formatting error. An example output of Results Object from immediate window:

image
image532×132 7.63 KB

5

@E_Metin

are the extracted values correct?

or arethey wrong?

because ocr confidence can be full as well if its clear

cheers

6

Some of the extracted values are wrong.

7

Hey @E_Metin
Confidence rate doesn’t mean the extracted value will be definitely correct. But it is a good sign for your solution. Think to add some cross check or business rules.

8

Thanks, there are existing regex validations but we are getting false positives which can only eliminated by checking extraction scores.
This problem does not happen on Automation Cloud. Only on On-prem Automation Suite installation. AI center version is 2024.10. We will be raising a ticket to UiPath support.