Is it Possible to Automate Java SWT applications using Uipath?

Help Activities
, ,
1

Is it Possible to Automate Java SWT applications using Uipath? Because I am facing issue in this applications sometimes, Uipath select the whole window. But i need to select the particular position.

Note : Java extension is enabled and I tried with uiexplorer (Default,AA,UiAutomation) options also.

2

Hello,

Java Extension is not used for SWT applications, since the stuff is drawn on the screen using some native libraries. The only information you can obtain is generally the one exposed by AA and UiAutomation. In those cases, Computer Vision is your best alternative.

Some things to try, before moving to CV are:

  • Try to get to the elements going down the hierarchy, and see if you can get them, maybe there is a problem with exposing the position on the screen.
  • Try to use ClickText / ScreenScraping to interact with the elements on the screen.
3

hi @vlad.coteanu already I tried with screen scraping and click text. but it is not working.

4

Then the only option left is Computer Vision.

5

Hi @vlad.coteanu now i am try with CV. I am try with CV screen scope and indicate the Java swt application. but it is not indicate properly. I am receive the error like Response from server is not valid.

6

Does this help: Find CV Text: Response from server is not valid (good internet connection & no proxy) - #9 by glenn.robinson?

7

@vlad.coteanu I tried with Cv also. but it is not working so, any other option is available in uipath.

8

Hi @BNK ,

I agree with @vlad.coteanu - this has worked for me in the past while working with Java applications:

On each selector - open in UI explorer and navigate through the hierarchy on the left side of the screen:

image
image694×773 33.8 KB

This will help you identify more elements than just the basic selector can.

Another possible solution would be to use Anchor base activity (Activities - Anchor Base) if you can identify some stable elements next to elements that you want to interact with.

Also make sure you install the Java bridge (Studio should detect that you are trying to automate Java and suggest this automatically). LATER EDIT: not valid suggestion, because Java bridge is part of the Java extension which is not used for Jawa SWT applications (see first comment from @vlad.coteanu )

Constantin

9

@Constantin_Dumitrascu I don’t know, How install java bridge.so, Please guide me.

10

Hi @BNK ,

Apologies, Java Bridge is part of the Java Extension and I just re-read the comment from @vlad.coteanu which states:

so this suggestion is not valid. I have corrected my previous post to specify this fact.

Constantin

11

Hi @vlad.coteanu @Constantin_Dumitrascu
In Java Desktop application(Java SWT) I have One window like Chess (Square boxes). I need to select one of the square box but Uipath select the Whole window. I tried with click text, screen scrapping,Cv. but it is not work. So any other options available for select a single square box.

12

Hi @vlad.coteanu:
In Java Desktop application(Java Swt) . I have a window like chess(square boxes). I need to select the One of the box but now it is selecting the whole window. so any technic is available for choose a single square box.

13

I have no idea… I am not sure what are the limits of CV. Can you add a picture and tell us what exactly you want to do with that squares? (Extracting text, clicking them)

14

download Hi @vlad.coteanu I am worked on My client machine. So, I am not able to share the screen shot . I need to click the square box . so for example i attach the chess image. In that window i need to choose one box. but now it is selection whole window.

15

Hi @vlad.coteanu
<wnd cls=‘SWT_Window0’ idx=‘23’ /
<ctrl role=‘client’ /

Where ever i click the inside the window i am receiving this information only. I am not able to differentiate the square boxes.

16

Maybe some mathematic calculations can help to click the correct pixel. For example, you know how many chess squares are on a line and on a column. You can get the position on the screen, in pixels, of the whole window (GetPosition Activity).

You can use Click with offset, starting from the top left corner of the “client”.