What are the differences between Default, AA, and UIA-selectors in the property panel, best practices for UI framework selection, and the combined impact of Input Method choice?

Help Studio
, ,
1

Hi! This post includes three questions because the answer to each is presumably affected by the answer of the other two.

Regarding question one. As a novice, I thought that “Default” was an option that automatically selected between AA or UIA, but I’ve learned that is not the case, as seen by the example selectors below from an application when using Click (modern). Note that I put in wildcards to not disclose organisation details.

* Default - "<wnd cls='*' title='*' /><wnd ctrlname='ControlAxSourcingSite' /><wnd ctrlname='scrollContent' /><wnd ctrlname='TopControl' /><wnd ctrlname='_TopControl_Toolbars_Dock_Area_Top' /><ctrl name='DockTop' role='grouping' /><ctrl name='Toolbar' role='tool bar' /><ctrl name='*' role='push button' />"
* AA - "<ctrl idx='1' name='DockTop' role='grouping' /><ctrl name='Toolbar' role='tool bar' /><ctrl name='*' role='push button' />"
* UIA - "<uia automationid='Toolbar : UltraToolbarDocumentation' name='UltraToolbarDocumentation' role='tool bar' /><uia automationid='[Toolbar : UltraToolbarDocumentation Tools] Tool : Cancel - Index : 0 ' name='*' role='button' />"

Therefore, I now I have type into and click activities that use either Default, AA and UIA when my intention was to use only AA or UIA. In the example above, it’s obvious which one is UIA, but depending on the specific element or app I use, the Default and AA selectors look very different and I can’t determine which is which and have to resort to re-indicate each activity. I’d like to avoid that. Is there a single attribute that is unique to either with which I can reliably identify if an activity is using the Default or AA framework?

As for my second question, is there any best practice for determining the most suitable UI framework? The answer I am expecting is something along the lines of “Always try X first in all cases. If it doesn’t work, try Y, and if that doesnt work, use Z. The only exception is A, where you should always use Y from the start.”

As for my third question, does the choice of Input Method affect the reliability of the selected UI framework?

  • Default + Hardware events
  • Default + Window messages
  • Default + Simulate
  • AA + Hardware events
  • AA + Window messages
  • AA + Simulate
  • UIA + Hardware events
  • UIA + Window messages
  • UIA + Simulate
2

Hi @Cedrik ,

  1. Default: This option is the default selector used by UiPath Studio. It combines multiple methods, such as UI Automation, Accessibility, and others, to identify UI elements. It is recommended to use “Default” selector if other options do not work, as it provides a fallback mechanism.

  2. AA (Accessibility Automation): This option uses the Accessibility framework to identify UI elements based on their accessibility properties. This is particularly useful when automating applications that are designed to be accessible for users with disabilities, and may not have proper UI Automation support.

  3. UIA (UI Automation): This option uses the UI Automation framework, which is a Microsoft technology, to identify UI elements based on their properties, patterns, and relationships. It provides more precise and reliable identification of UI elements in Windows-based applications.

Best practices for UI framework selection in UiPath:

  1. Use “Default” selector as the first option, and only consider other options if it does not work.

  2. Use “AA” selector when automating applications that are designed to be accessible, and do not have proper UI Automation support.

  3. Use “UIA” selector for Windows-based applications, as it provides more precise and reliable identification of UI elements.

The combined impact of input method choice depends on the application being automated, type of action performed. In general, the “Default” input method is recommended, as it automatically selects the best input method based on the target application. However, in some cases, using other input methods like “Simulate Type/Click” or “SendWindowMessages” may be necessary for improved accuracy and reliability.

Regards,

1 Like
3

Thank you for your reply to question 2 and 3! Much appreciated and helpful. Could you clarify the following so I can better understand and follow your advice?:

  • Where can I find documentation about what specific methods “UI Framework: Default” utilises and details about the “fallback mechanism”?
  • Where can I find documentation mentioning that the “UI Framework: Default “automatically selects the best input method based on the target application”?
  • How do I reliably identify whether an application is “designed to be accessible”?
  • Does “Windows-based” refer to applications running on Windows 11/10/etc, or applications made by Microsoft?
  • Could you give a specific example from your experience with UiPath about how the appropriate Input mode (given the choice of UI Framework) depends on the target application and performed action?

Thank you in advance. If you happen to have an answer to question 1, I’d appreciate it too.

Note: there is no “Default” input mode, which you mentioned, when using the Modern Design Experience and Click (modern). The input modes I can select from are Window Messages, Hardware Events and Simulate.

4

Hi,

I too tried to look for documentation about the UI frameworks, but I found just this link from UiPath docs:
https://docs.uipath.com/studio/standalone/2022.10/user-guide/uipath-explorer#:~:text=selector%20is%20valid.-,UI%20Frameworks,-Changes%20the%20technology

There’s also this if you want to learn about AA/UiA: UI Automation and Microsoft Active Accessibility - .NET Framework | Microsoft Learn - https://learn.microsoft.com/

In a nutshell: Try Default first, but if it doesn’t work test the other ones :slight_smile: From my experience, I haven’t had to change it often… but for some applications all selectors haven’t been detectable with the Default option (eg. MS AX).

Regarding the Input mode, I think it’s mostly about experimenting which works. I usually try in the order Simulate → Window messages → Hardware, because of the speed and background support from the first two.

1 Like