UiPath and Java

uiautomation
p_qa

#1

Does UiPath works with Java interfaces?

I tried to send a click Image, click or text.into but I’m getting an error message from UiPath

The message appears a while after indicating the action in the screen. After I press ‘indicate on screen’ I can hover on the java interface but clicking on an item doesn’t pick it up.

Any ideas on what is the problem?


Error HRESULT E_FAIL has been returned from a call to a COM component
Java Applet window Autmation
JavaFX based desktop application support for automation?how?
#2

Hi @selrac,

Have you installed Java extension for UiPath? You can find it here :

Regards,
V


#3

Yes, I did, but it didn’t make any difference


#4

Hello!

How do I know if the Java Extensions is correctly installed and running?
I have clicked the Extension, it ran a “JavaScreenCrappe install” (on the background, no setup window appeared to me), but I don´t know if it is running ok.
I have a Java Windows application to automate, and UIPath cannot identify the fields (edit box) inside the application.

Thanks!


Java Application Automation
#5

Hi,

I have tested now in my home (with all Administrative rights) and the behaviour is the same.
Can someone confirm if this is actually a limitation of UIPath?

Here is how it can be simulated:
Download the public Java Windows application:
Sistema Público de Escrituração Digital
The application is a Brazillian Government software used by thousands of companies for fiscal bookkeeping.
Just install the application, open it, and follow the menu:


(…I know, terrible choice of colors for the menu)

A new window will appear with some fields. UIPath can’t select the fields.

I appreciate any help to make this work.

Regards


#6

#7

Thanks @badita , I am checking the troubleshoot right now.

At the same time you replied here, I have moved my posts to new thread… sorry for the mess.


#8

No need to open a new thread. Please repost here and I will delete the other.


#9

Please delete the other thread. Posts reactivated.

As for the troubleshoot, the java application has a custom JRE folder.
All the requirements in the troubleshoot are met:

The ProcessExplorer:

The acessibility.properties of the custom JRE:

The DLL inside “ext” folder of custom JRE:

And the DLL inside “bin” folder of custom JRE:

Note: I haven’t configured any of this manually. These evidence I have found already applied in my environment.

But still, UIPath can´t get the fields inside of the application.


#10

@qateam, @adrian


#11

Just to give more clarity, the standard JRE installation also has all UIPath DLLs (ext, bin) and configs (accessibility.properties).


Screenscraping JAVA Apps / GUI Failure "HRESULT E_FAIL has been returned from a call to COM component"
#12

I recently diagnosed a similar issue where those who developed the Java app intentionally disabled Java Active Accessibility from security reasons.

Does UiPath recognize anything or does it fail only on certain parts of the UI of your app? You can perform a simple test on the UI of Java Config application (in control panel).

To verify this you can install the standard Oracle Java Bridge and use Java Ferret and / or Java Monkey to see if Java Active Accessibility is supported by your app; see: http://docs.oracle.com/javase/accessbridge/2.0.2/setup.htm

Regards,
Adrian.


Java Applet window Autmation
#13

Hi @adrian thanks for the reply.

Some elements of the UI can get selected by UIPath (like the labels for each edit box). But the edit boxes itself can’t be selected (except very few disabled fields). I don’t understand why.

I was able to install the Java Bridge (copy the files and all), but my JavaMonkey/JavaFerret is not capturing any UI element (still investigating).

If you could be so kind to simulate using the Brazilian application (follow the steps I have provided), I would be grateful. I need to know if this application can be automated with selectors.

[EDIT]
I have just been able to make JavaMonkey work.
It only gets elements from other Java applications, but not from SPED application.

Does this indicate that the application cannot be automated through UI?
Still, why can some elements be selected from UIPath?


#14

What I have encountered in some Oracle Java based apps is:

  • transparent windows covering the edit boxes and other elements
  • to check if the edit boxes are there you need to open UiExplorer and dive into the UI Tree.
    If this is the case the selectors are reliable but you will not be able to use indicate on screen but get them manually from UI Explorer.

#15

@badita the UIExplorer can’t get the full UI Tree (it would be a blessing if it did. :slight_smile: cause then I could build the selectors manually). But the truth is it only retrieves a few elements (some panes and some labels).

If someone can confirm if this an actual constraint (to use selectors) I would appreciate the help.

Regards


#16

@qateam


#17

You can also use the search feature in UiExplorer. You can use a condition like and uncheck “children only”.

Adrian.


#18

@Thiago , @badita

I’ve run into a similar issue with a Java application. UiPath in this case does seem to recognize the UI tree of the app (using UIExplorer, verified using Java Monkey), but I can’t use the Record feature to easily select elements: it highlights the entire frame instead of individual text boxes, labels, etc. I can manually add selectors and events, but given the size of the application and the complexity of the UI tree this becomes impractical.

There are no transparent windows covering the edit boxes in my case that I can detect. It is an Oracle Webforms app (deployed as a web-enabled Java applet), would that factor into it somehow?

Did you find a way to fix your issue? I believe it could be related.

Using UiPath Community Studio, JRE 6 (x86), and Windows 7 (x64).


#19

If I understand it correctly your problem is related to selecting Java objects on the screen and you are still able to browse Java objects in UiExplorer tree.

The issue might occur if you have 2 displays with different resolutions and dpi. If that’s the case please try using only one display and see if the problem persists.

Adrian.


#20

Thank you for your swift response.

Yes, that is exactly my issue. However, we were only using one display, albeit with a low resolution. Also, the applet does open in its own desktop window (not contained within a browser tab). Would that cause issues?