Citrix Automation - Recognize whether button is enabled/disabled

Selvasathappan · August 27, 2018, 2:31pm

Hello Everyone,

I am trying to Find a record in a citrix application. However sometimes due to application issue(old application), it doesn’t return the result even if the element is present. But one good thing is if a record is found, it returns the result immediately and “Stop” button is disabled and the loading bar is also empty. On the other hand when the record is not found, the “Stop” button is enabled and loading bar shows progress in green color. Is there any way, I can use any activity to identify this? I already tried image exists activity, but it isn’t accurate as the differences between images are very minute. Hence for the same reason, I had to set a accuracy limit of 0.8-1. Please help

If a record is found, it should make some updates and move to the next record whereas if a record is not found, it should store that “Id” value which I typed in “Find” window. There are more than 1500 records. Please help. Any suggestions/approach would be very helpful. Please find the screenshots below

Please note that this find window doesn’t have any unique selector as it is part of a high level window(remember citrix automation)

carmen · August 16, 2018, 1:28pm

The clue in this case is the accuracy by default is 80% you should try to up if you wanna find the activate button.

Selvasathappan · August 16, 2018, 1:32pm

But Uipath accepts elements as present/identifies it even if it is 0.5. By setting upto “1”, don’t I risk a chance to miss valid/qualified records as well? I already tried it using “1” but it is missing the valid records as well.

loginerror · August 16, 2018, 1:32pm

Hi @Selvasathappan

I think in this case you will need to find the middle point of accuracy versus robustness.

Due to the minimal difference between the color of an active and not active button, you might want to/need to set higher accuracy to be able to detect that slight change.

Selvasathappan · August 16, 2018, 1:40pm

Is there anything which we can do with the “loading bar” ? What do you think? It has green color. but the problem is robot doesn’t throw an error when the record is not found, It just pastes(update which I was talking about) the record in Find window type field and proceeds to the net record. I have to find a way to stop this, so that I can capture the missing/not found records

amakundu · August 16, 2018, 1:43pm

I would like to compare both the image (Button and progress bar). for progress bar should be all grey for comparison.

Selvasathappan · August 16, 2018, 2:00pm

I tried OCR text. Do you think it is reliable approach? I used scrape region and found the text value which was something like “5mg” for disabled button and “Stog” for enabled button. But will it remain the same always? As I can’t go live with this kind of unreliability. Hence quite not confident with this

loginerror · August 16, 2018, 2:03pm

I think I would go with the progress bar.

I would start the search, give it a slight delay and then wait until the progress bar is completely grey. If it is not fully grey before the timeout, then it would throw an exception that you can handle.

Selvasathappan · August 17, 2018, 9:56am

Guys! Does On Element appear would be of any use here? Does citrix automation helps identify “Stop” as an element as I have indicated that on screen? Will experiment anyway. Is it a reliable approach or have any of you tried that before?

loginerror · August 17, 2018, 10:18am

I have one idea. Could you maybe try to look for any Accessibility Features in your Windows, such as high contrast color scheme?
Maybe it would make the Stop button’s states more distinct and therefore easier to detect?

Selvasathappan · August 17, 2018, 11:06am

Hey! Thanks for the response. Will try that one as well. However I had another idea, When I send hotkey “Tab”, it only passes to active element, It doesn’t send to “Stop” button as it is disabled. Is there way through which I can find out how many tabs is possible here, store it in a count variable Does it even make sense or is this even possible? Or I even know the no of tabs possible in the screen. Can this be used? Acually am not able to frame this one properly. Struggling to implement this idea.Thank you once again for response.

loginerror · August 17, 2018, 11:18am

Unless you can directly request a change to the application to get an extra status box display, I think you will need to work with what you have. In this case, an image of the search window.

Isn’t there a status line at the bottom of the app (or somewhere within its UI) that would provide some extra information?

Selvasathappan · August 17, 2018, 11:22am

Is there a way to extract the text of elements/buttons in citrix environment? I mean not OCR approach. For ex: I would like to retrieve the text of an element where the tab key is sent. I would like to run a for loop/while loop and get the list of elements. Is it possible ?

loginerror · August 17, 2018, 11:26am

With Find Image you might be able to detect the slight blue border of the button. If that works, you might give it a shot.

After you do the search, you can send a bunch of tab presses. If the stop button is not active, it will go 1 tab too far. This is where you should be checking for an image of active box/button. If it is over the stop button and is now farther and you can detect it is there, then it will work.

Madhavi · August 18, 2018, 10:20am

@Selvasathappan Use Image exists to check Active or Inactive button. As its citrix automation, your BOT will be able to differentiate images every for tiny differences. This will work.

Selvasathappan · August 18, 2018, 3:33pm

@Madhavi I tried that already with accuracy parameter between 0.8-1 but it seems to misbehave. As the difference between images are very minute, It doesn’t work as expected. Even for valid records, it takes the branch/route of a “Not found” record and performs the subsequent activities. Thanks for your response. Please do let me know if you have any other suggestions/approach

badita · August 19, 2018, 4:02pm

Since it is Citrix then only Image recognition works.

There may be another approach.

Click Button
If Expected Result (Window Popup, etc)
=> button was enabled and continue to do the A branch
Else
=> button was disabled, do something else

Selvasathappan · August 19, 2018, 9:40pm

Hello @badita,

Thanks for your response. As stated earlier, since there is a minute difference between button when enabled and disabled, Uipath isn’t able to identify it accurately. Is there a way to handle this? Or do you mean to say that since it citrix, Uipath will definitely identify the difference ? I tried already but it’s taking the else branch for valid records as qell

Selvasathappan · August 20, 2018, 12:32pm

@badita, @loginerror, @Carmen - Guys, I tried Scrape relative for “Search” field using citrix recorder option. When the button is enabled, the value it scrapes is “By Columns v” and while it is disabled , it scrapes something like “Ev columns”. So anything other than “By Columns v”, should go to else block. It’s working fine now. But need inputs from you on whether it is a reliable approach or not. I tried 10 -15 times, it returned the same values. So chose this approach

Any help is much appreciated. I am only concerned about reliability. Request you to share your views

loginerror · August 20, 2018, 1:34pm

Hi @Selvasathappan

It is amazing that you found the solution. Personally, I would design a stress-test workflow that would test this solution in a loop for a higher number of records, closer to a 1000 (if you can prepare such an input list).

I would definitely try to rule out the possibility of any glitches happening. For example (and I am a bit out of my depths here, so excuse me if I speak nonsense) it is theoretically possible that the OCR engine gets an update that makes it properly recognize the text from the disabled textbox.

But other than that, it looks like you found your solution