Studio Web UI Automation Public Preview

News Product News
, ,
1

UI Automation support in Studio Web for web applications automation

The Modern UI Automation experience is now available in Studio Web, offering you (almost) all the means for automating web applications.

UI Automation activities in Studio Web leverage the automation design-time experience offered by Studio and StudioX, but also benefit from Studio Web’s new activities configuration and just-in-time execution capabilities.

What can be automated with UI Automation activities in Studio Web

Chrome browser automation

You can build automations with any web application rendering in Chrome. If the target UI element is in the browser viewport, you can use the activities available in UI Automation to interact with it.

What cannot be automated (yet)

Studio Web aims to help you build cross-platform automations. Thus, automating native desktop applications is not supported. Both Studio and StudioX continue to offer the right tooling for desktop automation.
Another thing that is worth mentioning is that since automations run on a new instance on the cloud, you won’t be able to see the automations running on your local machine, as you are used to. More about that here, and also the last tip here.

No matter if we are talking about structured or unstructured data extraction, we do not support this scenario for now. But do not worry, we are working on it. However, getting a text from the screen is possible with the GetText activity.

Also, the recorder is not available for the moment. If you think we should prioritize bringing it here, let us know!

How can I find the UI Automation activities in StudioWeb?

Open Studio Web and create a new project or open an existing one. Click on the “+” button to add an activity. Search any of the UI Automation keywords, e.g. “browser” and select the activity you want to be added to the canvas.

image001
image001770×409 10.9 KB

Installing the extension for Chrome

To be able to interact with the browser, the UI Automation activities require the UiPath extension for Chrome. Thus, as the first step before you start designing your automation, you must install it.

It is straightforward: just add any UI Automation activity on the canvas and you will be guided to install the extension.

image003
image003692×451 21.6 KB

image005
image0051294×278 16.6 KB

Note, once you add the extension to Chrome, all your opened tabs will be automatically refreshed to make the them available for further automation.

If you have an old extension, e.g. 21.10, already installed, you do not have to remove it. Both extensions will appear in your browser and will continue to work properly.

image007
image0071676×39 2.7 KB

Anatomy of a UI Automation activity

All the UI Automation activities follow the same design and structure. The properties are available and editable inline:

  1. The scope of the activity. Usually, this is inherited from the UseBrowser scope activity
  2. The ‘Indicate on screen’ button to indicate the target UI element.
  3. The most used configuration properties available immediately.
  4. The advanced configuration properties section collapsed by default.
    image009
    image009691×836 36.4 KB

An extensive list with a short description of the available activities

  • Use Browser – defines the automation scope. Once the tab you want to work on is selected, all the subsequent activities will focus on UI elements within it, at both design time and runtime.
  • Click – clicks the target UI element
  • Type Into – types the text/secure text into the specified input field
  • Get Text – gets the text from a target UI element
  • Keyboard Shortcuts – sends a combination of keys to a target UI element
  • Select Item – selects an item from a dropdown list
  • Check State – verifies the state of the application by checking the existence of a specified UI element on the screen
  • Go To URL – navigates the browser tab selected as a scope to the specified URL
  • Check – checks or unchecks a checkbox
  • Navigate Browser – can close or refresh a tab, and navigate it back and forward
  • Hover – performs a mouse hover over a UI element
  • Take Screenshot – takes a screenshot of the specified UI element. If no target is defined, it takes a screenshot of the entire browser viewport.
  • Get URL – gets the URL of the scope browser tab

Tips for building automation with the web version of the UI Automation Modern activities

  • Most of the UI Automation activities require a browser tab as their automation scope. Setting a scope guides you at design time, but also improves the execution performance. Start by adding a scope for your automation. Search the Use Browser activity, add it to the canvas and select the tab you want to automate. The tab must be already opened to appear in the ‘Work in tab’ dropdown.
    Note: you cannot indicate a tab, only selecting it from the existing opened tabs will work.

    image011

  • In case you want to work with two or more tabs at the same time, card in card functionality is available. The activities will inherit the inner most scope, out of the box, but can be changed if required.

    image012

  • If placed outside a specific browser scope, the Take Screenshot activity captures the entire browser window

  • The supported input modes are ChromiumAPI (default) and Simulate. Switch between them as you see appropriate.

  • Remember that the automation is executed in the cloud, on a fresh operating system instance and Chromium browser instance is spawned just in time. Thus, at runtime, the browser has no cookies set like you might have on your own machine, at design time. Accepting cookies for the target application should be coded as part of your workflow.

Troubleshooting failing jobs

The simplest way of troubleshooting a failing job with the current support is by taking screenshots and uploading them into your Google Drive or Office365 One Drive/Sharepoint folder. You may also consider sending them to your email account. The screenshots will help you understand the UI state at certain times when a job fails with a ‘UIElement no found’ exception.

We know that’s not ideal and we have something really nice coming up :wink:

Demo video

A few known issues

There are still a few known bugs and limitations that made it to the release. We’re planning fixes for them really soon, so hang tight :pray:

  • Right after installing the extension, if trying to automate a tab that is already open, there may be some lag in the functioning of the selection screen. It is recommended to open (or refresh) the targeted tab after installing the extension.
  • Type into activity – adding an Enter key (through the context menu) to the typed string, will remove any variables present in the “Type this…” field.
  • Keyboard shortcuts activity – adding shortcuts directly in the Shortcut field (instead of through the “Add shortcut key” button) will show a validation error.
  • Click activity – the “Win” modifier key is present in the modifier keys list, even though technically it cannot be executed

Enjoy automation and please share your feedback!

Thank you,
Gheorghe

14 Likes
3

Good one must try !

Will update my feedback soon.

2 Likes
4

Nice One would be trying it soon

1 Like
5

Great News, will be trying it soon.

6

This is good!!!

7

It would be even better if the data in Excel file could be quoted

8

How can i indicate a click element in (file upload popup) to upload file for a browser from the pop up in studio web??/
Any solutions please?