UiPath Community Stable Release 2022.4 - UI Automation, Computer Vision, SAP automation
This topic goes in-depth about the improvements in UIAutomation, Computer Vision, and SAP Automation. To read about other products, please navigate to the main topic here.
General
Chrome UIAutomation on Linux docker containers
An improved version of
Chrome Manifest V3 extension
We are releasing the brand new UiPath Chrome extension based on Chrome’s manifest V3 standard. It’s compatible with the previously released UIAutomaction.Activities packages are still under support, and we encourage you to install it as soon as possible and start using it within your automation.
Modern UIAutomation
For each UI element
This new activity’s main function is to help with simple scenarios like “click on every checkbox in this list”, or more complex scenarios like “for each product on this page, go to its product page, and perform some actions there”. And of course, it allows using complex filters, in case you don’t want to perform the action on every single element, but just on some of them, based on some rules.
Or in more technical terms, For each UI element loops over a set of elements, so you can perform any action on them (Click, Type, Get text, etc).
Drag & drop activity
This new activity enables the common interaction of dragging an object over a target. To configure it, simply indicate the source and destination elements – the activity will take care of the rest.
Scroll and switch-tab in the Selection Screen
A long-requested and super useful feature finally made it in 22.4: the ability to scroll the target page while in the selection screen, so you can easily find the targeted element without having to pause (F2) the selection. Also, switching the active tab using Ctrl-Tab is now supported 
New Visibility enforcement for elements
With this release there is a new option for enforcing the visibility of an element, either when interacting with it (in Click/Type activities), or when checking its state (in Check app state).
The previous visibility verification was also renamed to a more appropriate name (Interactive), and it will work just like before, ensuring the targeted element is in the DOM and can be interacted with, even if it’s scrolled below the fold, or even minimized.
The new option, Visible, will ensure that the element is actually rendered on the screen and would be visible to a human.
Store variables’ default value in Object Repository elements
When working with descriptors that contain variables, editing and validating them in Object Repository was difficult because variables have real values only at runtime. With 21.10, if variables have a default value defined, it will also be stored in the Object Repository element, to enable live editing and real-time validation in Studio.
Inject JS script activity
The Modern framework now has the capability of executing custom JavaScript code inside the web page you are automating. Pretty powerful stuff, use it wisely 
Take Screenshot also works outside a scope & without a target
Now you can use Take Screenshot without an App/Browser card, and even without a target, so you can quickly screenshot another app, different from the current one, or just screenshot the whole app, respectively.
Mouse Scroll with modifier-keys
We added key modifiers for the Mouse Scroll activity, for situations where a modifier key alters the behavior of the scroll.
Table extraction: extract URL and/or images sources in regular HTML tables
Now when extracting tables we automatically extract the URLs of the elements (if there are any), with the option to remove them if they’re not needed.
Object Repository > Capture all elements – Report issue
For the Capture all elements feature we’ve added the ability to report any missing detections. This allows us to improve the detections, making the functionality work better for you.
Capture Elements – region selection
We’ve added a few improvements to the Capture elements window to make using it faster and better: now you can select just a region of the whole window to analyze, and we’ve improved the detection of the type of elements using ComputerVision.
Create variables from Object Repository objects when dragging over canvas
When creating activities by dragging an object from Object Repo to the canvas, now the relevant variables will be created automatically if the object was using any.
Runtime browser switcher
If you created a workflow using a specific browser, say Internet Explorer, and would later like to switch it to another browser like Chrome or Firefox, now you can do that using a simple dropdown in the Project Setting, called Runtime browser . Pick any of the available browsers, and make sure to test the workflow to verify everything works correctly.
Check app state as a Retry scope condition
With this latest release, you can use the Check app state activity as the condition of a Retry scope, in order to execute a set of activities in a loop until a specific application state is achieved.
Use app/browser audit logs
There is a new flag in Project Settings > UIA Modern > Robot logging > Log audit information , that allows you to disable the logging generated by the Use app/browser activities.
Classic UIAutomation
Support for Azure Windows Virtual Desktop client
You can now natively automate Azure virtual desktops, with UIAutomation activities and Azure Windows Virtual Desktops client, using the UiPathRemoteRuntime and the RDP extension.
Support of Embedded Edge (WebView2 native)
Embedded Edge components via WebView2 native can be now automated via standard HTML selectors. One example is the SAP Business Client, which embeds IE but also starting with v7.70 embeds Edge as well. Any UI automation steering SAP Business Client v6 embedding IE, will continue to work the same if you are migrating to SAP Business Client V7.70 and use the embedded Edge.
Enhanced Start Process activity
The activity was enhanced to be able to start a process asynchronous (default) and also, synchronously.
Salesforce Lightning reliable automation on Firefox
Selectors based on Salesforce UI-specific attributes are now available for Firefox as well. These attributes bring in increased selector reliability and resilience to UI changes.
Salesforce Combo boxes and Grouped Combo boxes
SelectItem activity can now be used with the Salesforce combo boxes and grouped combo boxes.
For example:
Java 16/17 support
We have added UI automation support for applications running on Java 16/17.
Computer Vision
CV Extract Table
You can now fully extract the content of a vertically scrollable table using the CV Extract Table activity. This option is activated via a new boolean Property in the Options section called Scroll.
-
There is a preset limit for total table height of 30.000 pixels. This can be overridden by editing the desired Extract Table activity in your .xaml and adding a new property MaxTableScrollHeightInPixels=“30000” with the whatever value is necessary for you.
-
Community users might consume allowed data per minute very fast with large tables.
-
Scrolling, as it will be visible at run-time, will continue beyond the footer area of the detected table as a speed optimization.
CV Selection Screen
- It is now a lot easier to indicate full windows when working with native applications. F3 will now toggle between Element, Window or Region selection modes.
- UI colors & icons in the Selection Screen are now aligned with the rest of the UI Automation activities.
Cell Targeting Using Row/Column Index
No need to press Shift when targeting a table cell – row/column index selection is available by default.
StudioX support
CV Activities are available in StudioX when using the “Show Developer” filter.
New Computer Vision AI Model
Higher accuracy
We’ve retrained the CV AI Model and upgraded the internal neural network with a new state-of-the-art architecture, outperforming its predecessor. All UI elements are now detected with higher accuracy, but the largest improvement is related to working with very large tables on 4K screens - 
therefore we highly recommend verifying any existing workflow using large tables on 4K screens, using the preview endpoint below.
New CV preview endpoint
We’ve created a new preview CV endpoint, primarily aimed at testing existing workflows on the new model that will be released in 22.4, as well as testing the improved model. We recommend you also pair this with the latest version of activities. This allows for a more predictable migration to a new CV AI Model by being able to safely validate the new UI element detection capabilities in the context of your own data.
We highly recommend switching to the preview endpoint when working with large tables on 4K screens:
- When working with projects created using previous versions of the UIA Activity Package: the target endpoint must be manually set to
https://cv-preview.uipath.comand the CV API Key must be input, either in both Computer Vision and OCR Project Settings or in both CV Screen Scope Server URL and UiPath Screen OCR Logon Endpoint properties. - When creating new projects using the latest UIA activities Package: the target endpoint must be set to
https://cv-preview.uipath.com, either in the Server value in Project Settings or in the CV Screen Scope Server URL property.
The preview endpoint will be active until the release of our 22.4 activity package. After this date:
- The new CV AI Model will be deployed to our default production endpoint,
https://cv.uipath.com, so please update any projects created or validated using the new AI model to point back to this default value for the endpoint. https://cv-preview.uipath.comendpoint will no longer be active.
Other activities improvements
Support for Del key in the CV Selection Screen
Hitting Delete in the CV Selection Screen now deletes the current selection for quicker workflow building.
Delay Screenshot After Scroll
The CV Extract Table activity now has a DelayScreenshotAfterScroll property which enables it to wait for a specific amount of time after performing a scroll action before taking a screenshot of the target table. This is useful for tables that load extra items when scrolling down.
Exceeding maximum height in pixels when working with Table Scroll
You will now get an exception when trying to extract data from scrollable tables over the MaxTableScrollHeightInPixels value: “Extract table has height higher than the limit of {0} set to guard against infinite scrolling. Please adjust limit or check that the result is the desired one.” There is a preset limit for total table height of 30.000 pixels. This can be overridden by editing the desired Extract Table activity in your .xaml and adding a new property MaxTableScrollHeightInPixels=“30000” with the whatever value is necessary for you.
SAP UIAutomation
New Expand SAP ALV Hierarchical Table activity
Introducing a new activity that allows identifying any cell inside SAP ALV hierarchical table. After the identification of the cell all typical UI activities can be performed, such as Click, Double Click, Get Text, Get Attribute or other. For more, please visit the documentation here.
New Feature “Enforce hard timeout for SAP WinGUI”
Sometimes there are situation in SAP WinGUI, when you want either to cancel the long running SAP activity or SAP becomes unresponsive/frozen/hanging or network/”connection lost” issue appears.
This new feature will ensure, that SAP execution will be enforced to terminate after a predefined timeout. The right parameter can be found in Project Settings under UIAutomation Classic.
If “Enforce hard timeout” is activated (“True”), the UiPath Robot will terminate the activity execution and will throw an error after the timeout specified in the activity target. You can catch and handle this error message in a Try & Catch activity.
This allows UiPath Robot to overcome the above-described uncertainties in SAP WinGUI automation.
Improve UX when SAP GUI Scripting in not properly configured
We have worked on UX improvements to allow user better to understand what is happening, when SAP WinGUI automation is not working correctly. In most cases the main reason was misconfiguration or not complete configuration of SAP API Scripting interface.
From now on, we are providing clear guidance, which settings are not correct, e.g.:
Object Repository for SAP WinGUI activities
From now on, all SAP WinGUI Activities are supported and work with Object Repository.
" System exception at initialization: Cannot communicate with the browser, please check the UiPath extension"