We are using UiPath Document Understanding for invoice processing. We have noticed that some vendor invoice layouts have remained the same, but Classification/Data Extraction confidence appears to have decreased over time.
Can changes in the OCR engine, Document Understanding packages, UiPath platform version, or Digitize Document configuration cause different OCR text or confidence results even when the invoice layout has not changed? Has anyone experienced similar behavior?
Hi Community,
We are using UiPath Document Understanding for invoice processing and have encountered a strange issue regarding data extraction confidence.
Background:
Question 1:
We’ve noticed that our Classification/Data Extraction confidence scores appear to have decreased over time. Can changes in the cloud OCR engine endpoints, background DU platform updates, or Digitize Document cloud configurations cause different OCR text or confidence results even when our local configurations haven’t changed? Has anyone experienced similar behavior?
Question 2:
If the exact same invoice PDF is processed using an older cloud session/state versus the current state, is it possible to receive completely different OCR text or downstream extraction confidence? What is the recommended best practice to compare historical and current Digitization/OCR output to determine if OCR shifts are causing the confidence drop? (e.g., extracting and parsing the DOM object).
Any guidance or troubleshooting steps would be greatly appreciated!
yes, this is possible even if your local packages and workflow haven’t changed, since Document OCR is a cloud service and the backend can change over time.
For troubleshooting, I’d compare the Digitize Document output from a historical run with a current run using the exact same PDF. Check the OCR text, page content, and bounding boxes/words before looking at the extraction confidence. If the OCR output itself has changed, that would explain the downstream confidence difference.
I’d also recommend saving the Digitize Document/DOM output from successful runs so you have a baseline for future comparisons. If the OCR output is identical but confidence has dropped, then the issue is more likely on the classification/extraction side rather than OCR.
If possible, testing the same document multiple times in the current environment can also help determine whether the result is consistent or varying between runs.
@rangaiah.thatiparthi Yes, this can happen even if your invoice layout and local packages are same.
Since you are using UiPath Document OCR, there can be some changes on the cloud OCR side also. So same PDF processed after some time may give slightly different OCR text or confidence.
I would first check the output from Digitize Document before looking into classifier/extractor.
Take one invoice where you are seeing this issue and compare:
with the older execution if you have those outputs saved.
Check mainly the fields where confidence dropped like invoice number, date, total, vendor name etc. Sometimes even a small OCR change like missing character, word split/merge or position change can affect extraction confidence.
Also OCR confidence and extraction confidence are not exactly same thing. Even if OCR looks fine, small changes in text/position can impact the extractor confidence.
For Q2, I dont think old cloud session can be considered as a proper baseline. Better option is to keep few sample invoices and save the Digitize output + extraction result/confidence. Then you can compare same documents whenever this issue happens again.
One more thing, check ApplyOcrOnPdf setting also, specially if these are native PDFs.
My first check would be:
Same PDF → Digitize Document → compare old and current DocumentText/DOM.
If you already see difference there, then most likely issue is before extraction and I would share those details with UiPath support for further check.
Thank you for your response!
Thank you,Mona for your response!