Check Image Recognition Accuracy

Help
1

Is there a way to check how accurate images are being recognized in Activities that use Image Recognition (Click Image, Find Image, Image Exists, etc.)?

Examples:

  1. If the Accuracy property is set to 0.8, but UiPath always detect the image with 100% of confidence, you could use this information to know how much higher the value could be set.
  2. Another example would be if the detection failed. You could use this information to determine how lower the Accuracy should be set to have a reliable detection.

This would make it easier to determine a reasonable value for the Accuracy property of image based activities.

Best regards,

Vinicius Marques

1 Like
2

@viniciussm

That was a great topic to be discussed, but I don’t think we have a way to do that. As per my tests using the above activities, when the selector is the one which we got while indicating the elements, there is a chance of recognizing the element for sure, I mean 100%. But if it is dynamic, we see issue for maximum number of times. Not sure if this helps you :slight_smile:

3

Sorry, I’m not sure I understood exactly what you meant. What I am suggesting is a way to see how accurate an image detection was after running the activity, considering that the selector was correctly set. Independent of the result of the recognition (image detected or not), we could have the accuracy available through logs something similar.

4

Hi @viniciussm,

Do you mean the below?
image

5

Hi @rmunro,

Not exactly. What I’m suggesting is a place where we could see how accurate UiPath was able to identify the image after running the Workflow, through logs or something.

Examples:

  1. In the image you sent the Accuracy is set to 0.8, but if UiPath always detect the image with 100% of confidence and you could see this information, you might be able to configure this property even higher.
  2. Another example would be if the detection failed. You could use this information to determine how lower the Accuracy should be set to have a reliable detection.

I believe this would improve the experience when using Image Detection.

6

@viniciussm
you asked regarding a so called trustlevel, get returned by UiPath activity and indicating on how good the detection is rated?

7

@ppr
Yes, that is it. Just a way to see how well the detection went.

8

I like this idea.

It would be useful as an out variable in Find Image, Click Image, etc. for logging purposes or for sorting low/medium/high level matches if the situation called for it.

I think it could also be useful as a standalone activity. Perhaps you need to check files for a certain image. Use this activity to get a percentage and then sort: All images that are 30% match or lower are discarded, images 31% - 75% are sent to a folder for human review, images above 75% are a success. This can be simulated now with multiple Find Image activities, but a single match percentage activity would make it cleaner.

I have some processes that identify different companies and pages on invoices and an image match percentage activity would help those.

@viniciussm if you want to toss this up on the Ideas forum I’d give it a vote.

1 Like
9

@viniciussm
Great, I just pushed the words int this direction as the approach is common e.g in Datacapture Software (Datacap, Captivity, SmartFix …). Here is such information used to route the not recognized field to the staff for manually validation. So by knowing this concept i derived my wording.

I do feel that your idea is a perfect match to go within this category Feedback - UiPath Community Forum section and can be prepared for the UiPath Team as Feedback, Idea, FeatureRequest …

However, you mentioned a cool point

1 Like
10

Thank you for you feedback @DanielMitchell and @ppr.

I have just created the topic on that category: Information about actual Accuracy on activities that use Image Recognition

2 Likes