Needing to find text that matches a variable value and click to the right of it in RDP

Help Studio
1

I have a process that inputs an amount via a variable into a field. I also have to find that value which will be in a list shown below in the balance column. After locating that I have to then click to the right of it in order to type the searched variable value into the cell adjacent to it under the Adjustment Balance column. I have tried all ocr methods to locate the value but am at a loss. I know after I find the position then I can offset the x axis to click into it but I must be missing something.

image
image1079Ă—450 55.2 KB

2

Adjustment Amount column to the right not Adjustment Balance column

3

Hi @Dylan_Rowe,

Is this an SAP application?

If yes, kindly check the selector using the UI Explorer. There will be a property that indicates the row number. Kindly update that with an Int32 variable that will be set as a counter.

Hope this helps.

Kind regards,
Kenneth

4

Hello @Dylan_Rowe

Welcome to UiPath community…!

Are you able to inspect the elements? If yes, you have to make the selector dynamic to loop through the rows. For that, you have to find the attribute that corresponds to the row number.

Thanks

5

Hi Kenneth,

Sadly it is not. It is virtual environment so I didn’t think I had luxury of generating normal selectors for it.

6

I am not sadly. I am inside of Remote Desktop session. Ui path is not installed on the remote machine so I attach to the window with ui path running from mine. I think I am stuck using OCR and Image recognition.

7

Hi @Dylan_Rowe

I suggest you try using the Computer Vision activities since you’re automating a remote environment. AI Computer Vision enables all UiPath Robots to see every element of an interface. You can easily build a vision-based automation that will run on most virtual desktop interface (VDI) environments—regardless of framework or operating system.​

CV Activities documentation: https://docs.uipath.com/activities/docs/about-the-computer-vision-activities

Kind regards,
Kenneth

1 Like
8

If that is the case, better to go with Computer vision activities than going with OCR activities.

Thanks

1 Like
9

Hi @Dylan_Rowe

If this does help to solve your problem. Kindly mark the solution so it will help others as well. Thank you.

Kind regards,
Kenneth

10

Sadly we don’t have access to cv :frowning: … I have tried scraping the screen and looping through the Rectangle variables but there are multiple occurrences of the string I am looking for on the screen. I can get the position of them and have tried to set the clipping region but have had no success with it clicking the right text.

Closest I have gotten was using Click OCR Text. I was storing the integer for the occurrence property in a variable and searching for an image that should appear if it is successful. If it is not successful I then increment the variable and have it try for the next occurrence. It works some of the time but is pretty buggy and unreliable.

11

@kennbalobalo I was able to implement Remote Desktop runtime which has helped a little.

12

Hi @Dylan_Rowe

Have you tried using Computer Vision activities like we suggest?

Kind regards,
Kenneth

13

Yeah, I mention above we do not have access to computer vision sadly :frowning:

14

I understand. You can use OCR activities too but that’s not that reliable. Else, you will need to install UiPath inside the remote environment.