How reliable is Check App State for image recognition and anchors?

Help Activities
, ,
1

Spinning off my earlier post about detecting which box is checked:

image
image778×210 48.3 KB

I realized I can use Check App State. I designate an image selector for the empty checkbox, and the label as the anchor also using image recognition. Initial tests appear reliable, and this is a simple solution. It’s not a big deal to make my code this static, as the form is always the same for this process and there aren’t that many checkboxes to detect.

Basically if the element is detected, the box is unchecked…and if the element is not detected then I assume it is checked. I can do a similar thing by looking for the blank underscore next to the label to determine if it is initialed or not.

So my question is, how accurate are image-based anchors and targets? Is it “smart” enough to understand that the physical positioning is what matters since it’s all images and there’s no underlying page structure (ie no real selectors)?

image
image573×164 47.3 KB

My next test is going to be with Computer Vision to see how reliably it identifies things.

2

It is hard to tell. As there are multiple squares/checkboxes in the image, resulting in “duplicate elements” being detected which would ultimately fail the process

3

Yup, you’re right. I tried it with a document where that box is checked and what it did was find the nearest unchecked box it could find.

image
image499×132 40 KB

I suppose what I could do is indicate the entire combo - the unchecked checkbox, empty line, and label - as the element to detect. If it’s not detected, assume it’s checked.

4

The entire combo might work, but I highly doubt its reliability.

5

Computer Vision makes the same mistake, because it’s trying to find the closest match rather than an exact match based on physical position. What’s throwing it off is that the text “Multiple-Party Account” is actually there twice even though one of them is part of a longer string.

1 Like