How to do the image based UI automation development?

Hi,

I am trying to automate some workflow for a GUI based windows application. In that application there is not enough selectors available to do automation and the selector attributes are not reliable enough. The value of the attribute changes every time.
Just because of that I started using image based automation. Even that seems to be not reliable, in case of multiple displays, extended monitors, change of screen size resolution etc.

Can someone suggest me a reliable method to do GUI windows application automation? or how a reliable development is carried out with UiPath?

Any suggestion would be appreciated.

Thanks,
John

1 Like

Did we try with COMPUTER VISION activities

Cheers @John_D

1 Like

Thanks for the quick response @Palaniyappan

Yeah. This seems to be good idea. But I think that will increase my test execution time drastically :sweat_smile:
I am looking whether there is any way for a reliable and faster (test execution) solution.

An experienced person like you should have some solution on this kind of case.

1 Like

Fine
We got many options to handle such image based environments buddy
Like
—the most reliable one I use is SEND HOT KEYS activity with key as tab or enter to navigate across the fields
We need to ensure only that field remains in same position and we don’t need worry about the attribute that keeps changing
This hot key will work even in background

Or

Then I would prefer COMPUTER VISION why I prefer this as second choice is thought it’s more reliable and amazing feature that uipath gives us it needs uipath to be with appropriate version while if we have 2017 and 2018 version we won’t be able to use them but still it’s more reliable comparatively to Send hot keys

Cheers @John_D

1 Like

I tried as you suggested, the Computer Vision. It increased my execution time by four times.

Sending hot keys are an superb idea, i appreciate that but that may make a good solution in a longer run. Do you agree with me?

CV is a well and for this situation but timing is the only problem with that.

1 Like

Of course it would buddy
Please go ahead

@John_D

@Palaniyappan Oops i wanted to say that that idea may not make a good solution in a longer run.

1 Like

No it would work for sure even in longer terms unless the UI and elements arrangement doesn’t change
Like if we have now used two tabs to reach a field and if that field is moved later a bit and if we need two more tab keys only then we will face problem
Mostly that won’t happen
And even that happens and if have used selector based accessing anyhow we need to change the selector again…right
For that we can better use the reliable one until changes come
Cheers @John_D

@Palaniyappan, I can understand. I will let you know if anythings pops up.

I thought of making a solution with one set of method. But it seems like we need combination of different activities for different scenarios.

Once again thanks a lot man! :+1: Much appreciated!

1 Like

Sure
@John_D

Is Comupter Vision not available for Studio licensed users?