UiPath Community 2022.4 Stable Release - UIAutomation, Computer Vision & SAP Automation

UiPath Community Stable Release 2022.4 - UI Automation, Computer Vision, SAP automation

This topic goes in-depth about the improvements in UIAutomation, Computer Vision, and SAP Automation. To read about other products, please navigate to the main topic here.


Chrome UIAutomation on Linux docker containers

An improved version of

Chrome Manifest V3 extension

We are releasing the brand new UiPath Chrome extension based on Chrome’s manifest V3 standard. It’s compatible with the previously released UIAutomaction.Activities packages are still under support, and we encourage you to install it as soon as possible and start using it within your automation.

Modern UIAutomation

For each UI element

This new activity’s main function is to help with simple scenarios like “click on every checkbox in this list”, or more complex scenarios like “for each product on this page, go to its product page, and perform some actions there”. And of course, it allows using complex filters, in case you don’t want to perform the action on every single element, but just on some of them, based on some rules.

Or in more technical terms, For each UI element loops over a set of elements, so you can perform any action on them (Click, Type, Get text, etc).

Drag & drop activity

This new activity enables the common interaction of dragging an object over a target. To configure it, simply indicate the source and destination elements – the activity will take care of the rest.

Scroll and switch-tab in the Selection Screen

A long-requested and super useful feature finally made it in 22.4: the ability to scroll the target page while in the selection screen, so you can easily find the targeted element without having to pause (F2) the selection. Also, switching the active tab using Ctrl-Tab is now supported :v:

New Visibility enforcement for elements

With this release there is a new option for enforcing the visibility of an element, either when interacting with it (in Click/Type activities), or when checking its state (in Check app state).
The previous visibility verification was also renamed to a more appropriate name (Interactive), and it will work just like before, ensuring the targeted element is in the DOM and can be interacted with, even if it’s scrolled below the fold, or even minimized.
The new option, Visible, will ensure that the element is actually rendered on the screen and would be visible to a human.

Store variables’ default value in Object Repository elements

When working with descriptors that contain variables, editing and validating them in Object Repository was difficult because variables have real values only at runtime. With 21.10, if variables have a default value defined, it will also be stored in the Object Repository element, to enable live editing and real-time validation in Studio.

Inject JS script activity

The Modern framework now has the capability of executing custom JavaScript code inside the web page you are automating. Pretty powerful stuff, use it wisely :wink:

Take Screenshot also works outside a scope & without a target

Now you can use Take Screenshot without an App/Browser card, and even without a target, so you can quickly screenshot another app, different from the current one, or just screenshot the whole app, respectively.

Mouse Scroll with modifier-keys

We added key modifiers for the Mouse Scroll activity, for situations where a modifier key alters the behavior of the scroll.

Table extraction: extract URL and/or images sources in regular HTML tables

Now when extracting tables we automatically extract the URLs of the elements (if there are any), with the option to remove them if they’re not needed.

Object Repository > Capture all elements – Report issue

For the Capture all elements feature we’ve added the ability to report any missing detections. This allows us to improve the detections, making the functionality work better for you.

Capture Elements – region selection

We’ve added a few improvements to the Capture elements window to make using it faster and better: now you can select just a region of the whole window to analyze, and we’ve improved the detection of the type of elements using ComputerVision.

Create variables from Object Repository objects when dragging over canvas

When creating activities by dragging an object from Object Repo to the canvas, now the relevant variables will be created automatically if the object was using any.

Runtime browser switcher

If you created a workflow using a specific browser, say Internet Explorer, and would later like to switch it to another browser like Chrome or Firefox, now you can do that using a simple dropdown in the Project Setting, called Runtime browser . Pick any of the available browsers, and make sure to test the workflow to verify everything works correctly.

Check app state as a Retry scope condition

With this latest release, you can use the Check app state activity as the condition of a Retry scope, in order to execute a set of activities in a loop until a specific application state is achieved.

Use app/browser audit logs

There is a new flag in Project Settings > UIA Modern > Robot logging > Log audit information , that allows you to disable the logging generated by the Use app/browser activities.

Classic UIAutomation

Support for Azure Windows Virtual Desktop client

You can now natively automate Azure virtual desktops, with UIAutomation activities and Azure Windows Virtual Desktops client, using the UiPathRemoteRuntime and the RDP extension.

Support of Embedded Edge (WebView2 native)

Embedded Edge components via WebView2 native can be now automated via standard HTML selectors. One example is the SAP Business Client, which embeds IE but also starting with v7.70 embeds Edge as well. Any UI automation steering SAP Business Client v6 embedding IE, will continue to work the same if you are migrating to SAP Business Client V7.70 and use the embedded Edge.

Enhanced Start Process activity

The activity was enhanced to be able to start a process asynchronous (default) and also, synchronously.

Salesforce Lightning reliable automation on Firefox

Selectors based on Salesforce UI-specific attributes are now available for Firefox as well. These attributes bring in increased selector reliability and resilience to UI changes.

Salesforce Combo boxes and Grouped Combo boxes

SelectItem activity can now be used with the Salesforce combo boxes and grouped combo boxes.

For example:

Java 16/17 support

We have added UI automation support for applications running on Java 16/17.

Computer Vision

CV Extract Table

You can now fully extract the content of a vertically scrollable table using the CV Extract Table activity. This option is activated via a new boolean Property in the Options section called Scroll.

  • There is a preset limit for total table height of 30.000 pixels. This can be overridden by editing the desired Extract Table activity in your .xaml and adding a new property MaxTableScrollHeightInPixels=“30000” with the whatever value is necessary for you.

  • Community users might consume allowed data per minute very fast with large tables.

  • Scrolling, as it will be visible at run-time, will continue beyond the footer area of the detected table as a speed optimization.

CV Selection Screen


  • It is now a lot easier to indicate full windows when working with native applications. F3 will now toggle between Element, Window or Region selection modes.
  • UI colors & icons in the Selection Screen are now aligned with the rest of the UI Automation activities.

Cell Targeting Using Row/Column Index


No need to press Shift when targeting a table cell – row/column index selection is available by default.

StudioX support

CV Activities are available in StudioX when using the “Show Developer” filter.

New Computer Vision AI Model

Higher accuracy

We’ve retrained the CV AI Model and upgraded the internal neural network with a new state-of-the-art architecture, outperforming its predecessor. All UI elements are now detected with higher accuracy, but the largest improvement is related to working with very large tables on 4K screens - :exclamation: therefore we highly recommend verifying any existing workflow using large tables on 4K screens, using the preview endpoint below.

New CV preview endpoint

We’ve created a new preview CV endpoint, primarily aimed at testing existing workflows on the new model that will be released in 22.4, as well as testing the improved model. We recommend you also pair this with the latest version of activities. This allows for a more predictable migration to a new CV AI Model by being able to safely validate the new UI element detection capabilities in the context of your own data.

:exclamation: We highly recommend switching to the preview endpoint when working with large tables on 4K screens:

  • When working with projects created using previous versions of the UIA Activity Package: the target endpoint must be manually set to https://cv-preview.uipath.com and the CV API Key must be input, either in both Computer Vision and OCR Project Settings or in both CV Screen Scope Server URL and UiPath Screen OCR Logon Endpoint properties.
  • When creating new projects using the latest UIA activities Package: the target endpoint must be set to https://cv-preview.uipath.com, either in the Server value in Project Settings or in the CV Screen Scope Server URL property.

:exclamation: The preview endpoint will be active until the release of our 22.4 activity package. After this date:

  • The new CV AI Model will be deployed to our default production endpoint, https://cv.uipath.com, so please update any projects created or validated using the new AI model to point back to this default value for the endpoint.
  • https://cv-preview.uipath.com endpoint will no longer be active.

Other activities improvements

Support for Del key in the CV Selection Screen

Hitting Delete in the CV Selection Screen now deletes the current selection for quicker workflow building.

Delay Screenshot After Scroll

The CV Extract Table activity now has a DelayScreenshotAfterScroll property which enables it to wait for a specific amount of time after performing a scroll action before taking a screenshot of the target table. This is useful for tables that load extra items when scrolling down.

Exceeding maximum height in pixels when working with Table Scroll

You will now get an exception when trying to extract data from scrollable tables over the MaxTableScrollHeightInPixels value: “Extract table has height higher than the limit of {0} set to guard against infinite scrolling. Please adjust limit or check that the result is the desired one.” There is a preset limit for total table height of 30.000 pixels. This can be overridden by editing the desired Extract Table activity in your .xaml and adding a new property MaxTableScrollHeightInPixels=“30000” with the whatever value is necessary for you.

SAP UIAutomation

New Expand SAP ALV Hierarchical Table activity

Introducing a new activity that allows identifying any cell inside SAP ALV hierarchical table. After the identification of the cell all typical UI activities can be performed, such as Click, Double Click, Get Text, Get Attribute or other. For more, please visit the documentation here.

New Feature “Enforce hard timeout for SAP WinGUI”

Sometimes there are situation in SAP WinGUI, when you want either to cancel the long running SAP activity or SAP becomes unresponsive/frozen/hanging or network/”connection lost” issue appears.

This new feature will ensure, that SAP execution will be enforced to terminate after a predefined timeout. The right parameter can be found in Project Settings under UIAutomation Classic.

If “Enforce hard timeout” is activated (“True”), the UiPath Robot will terminate the activity execution and will throw an error after the timeout specified in the activity target. You can catch and handle this error message in a Try & Catch activity.

This allows UiPath Robot to overcome the above-described uncertainties in SAP WinGUI automation.

Improve UX when SAP GUI Scripting in not properly configured

We have worked on UX improvements to allow user better to understand what is happening, when SAP WinGUI automation is not working correctly. In most cases the main reason was misconfiguration or not complete configuration of SAP API Scripting interface.

From now on, we are providing clear guidance, which settings are not correct, e.g.:

Object Repository for SAP WinGUI activities

From now on, all SAP WinGUI Activities are supported and work with Object Repository.

Hello, we are having problems with Chrome extension (so far we know about Chrome, but not tested on other browsers) after update to 2022.4 all processes at 2 non-related machines stop working with Chrome (version 100.0.4896.127 , one is 32bit, rest is 64bit).
The error is same at all machines:
System exception at initialization: Cannot communicate with the browser, please check the UiPath extension
At machine 1: It turns out that the extension in Chrome was off, so solved by turning it on. So probably after the update the extension needed re-activation? Weird…
At machine 2: Installation seems okay, but the extension is not visible in Chrome Extensions.

I will update you in case we found out any more details but I wonder if we are the only there struggling with this. Roman

Update: After update on my dev machine - no issues at all. Machine 1 is installed in Windows Service mode. Machine 2 - hard to say :smiley:

Hi, we have the same problem here, we installed on a new computer a community version and once we try to launch the process, even if we added through Studio the Chrome extension, and even if we uninstall and install the extension, it does not work :confused: " System exception at initialization: Cannot communicate with the browser, please check the UiPath extension"

Hello @Hiba_B !

It sounds like there is a configuration issue with the Chrome extension on your machine.
Please check the troubleshooting guide from here:
Chrome Extension Troubleshooting (uipath.com)
It that does not help, then let’s connect via Zoom and debug this issue on your machine.


Ok, going to try and I will let you know :grinning_face_with_smiling_eyes:

Thanks a lot! This coincided with my issue with 22.4. Not solved yet.

Thank you!