How to use the Automation Language AutoIt in Combination with UiPath

It seems to be a litte bit as an antagonism to use another desktop automation solution in the context of UiPath. But my perspective is that each integration scenario is allowed as long you reach your final target fast and secure. In this post I describe how to use AutoIt in the context of UiPath, to complement particular activities which you can’t use easily otherwise. In this consideration I refer to my question about the possibility of recording the X and Y position of the mouse pointer with the UiPath recorder. Here I just wanted to record my mouse movements on the canvas of Microsoft Paint.

AutoIt is a freeware BASIC-like scripting language designed for automating the Windows GUI and general scripting. It contains AutoItX, a DLL version of AutoIt that provides a subset of the features of AutoIt via a COM interface. We can very easily install a package of AutoItX from the NuGet Gallery via Manage Packages.

To test it we implement a very easy test via an Invoke Code activity. We move the mouse pointer from its actual position to 0, 0 with the slowest speed.

002

If we run that test we experience a surprise, an exception. :face_with_raised_eyebrow:

003

The error message told us that it can’t find the dynamic link library AutoItX3.dll. To correct this error we copy this library from build directory to lib directory. We must do this in the .nuget\packages directory of the %HOMEPATH%, in my case Users\root. If we run this test again now, it will work fine. The mouse pointer moves slowly from its actual point to the position 0, 0 on the screen. Good :slightly_smiling_face:

In the next step we use the AutoIt recorder. But at first we must download release 3.3.14.0 from AutoIt archive and unpack from zip archive the files Au3Record.exe and ASWhook.dll from directory Extras/Au3Record. Unfortunately is the recorder not available in the actual releases of AutoIt anymore. Now we start the recorder an record any activities on the canvas of Microsoft Paint.

We get a list of mouse moves which we copies from the recorder to a Invoke Code activity from UiPath. All we have to do now is to set the prefix AutoIt.AutoItX- in front of every command. You can do this very fast via an replace command in your favorite editor. Replace CrLf with CrLfAutoIt.AutotItX. Now only the first line we must change manually.

006

This Invoke Code activity is part of the following workflow. It contains the Start Process activity to open Microsoft Paint and a sequence to choose the pencil and the color. Then the strokes are painted on the canvas via AutoItX. After that we choose in another sequence text and color and then we write via AutoItX a text on the canvas.

007

And here the result.

Great, all works as expected :smiley:

The combination of UiPath and AutoItX offers us the possibility to record mouse activities on elements which we can’t reach with UiPath at the moment. In my opinion it is not bad to use different automation solutions together if they help us quickly to come to a solution. The basic concept of UiPath allows us this possibility, we should not be afraid to use it.

Addendum 10.09.2022: The current AutoItX3.Assembly.dll from release 3.3.16.0 is only available for the target platform dotNETFramework, Version=v4.0. The approach presented here is deprecated, because it can’t work in the Windows compatibility mode. It is suggested to use the approach with the AutoIt activity.

9 Likes

Hi,

I have been using Autoit for the past 6 years, automating business processes and have only just found out about RPA and UiPath and I am extremely excited, I am doing the Academy now to learn UiPath so I can move into a RPA Role (I dont think companies know/consider AutoIT an RPA type program).

I am amazed at what I have found, the community is great and the product seems fantastic! Its good to know my AutoIT skills will work in harmony with this new automation tool!

1 Like

@TomWebb

Hello Tom,
welcome in the UiPath Community.
Great to hear that you want to combine your consolidated automation scripts with UiPath.
Let us know in due course your experience, especially your integration scenario with the orchestration.
Thanks and best regards
Stefan

1 Like

Hi @StefanSchnell ,

This is great! Thank you so much.

Could you please let me know how you handle the prefixes for/how you could declare the following the line of code:

_WinWaitActivate(“Cortana”,“”)

1 Like

@Syeda_Kunza

Hello Syeda,
welcome in the UiPath Community.
The _WinWaitActivate is an UDF (User Defined Function) and it is not possible to use this kind of functions with AutoItX. You can use UDFs only with AutoIt.
Best regards
Stefan

1 Like

@Syeda_Kunza

Hello Syeda,
you can find a solution here to use the fully range of AutoIt.
Best regards
Stefan

It is wonderful and it worked with me excellently, but do all the commands work, for example I want to call through Autolt with adb commands to communicate better and faster. This is an example of how to contact, but it did not work with me

#include <GDIPLus.au3>
#include “ImageSearch.au3”
$port=‘62025’
;click
RunWait(@ComSpec & ’ /c adb -s 127.0.0.1:’ & $port & ’ shell input tap 445 914’, “”, @SW_HIDE)
MsgBox(0,0,0)

This command in AutoIt communicates very quickly and the phone has been controlled faster. Is it suitable for it to work from within the uipath because the communication system in the mobile through the uipath program is very slow and has many problems

Hello, I have read many explanations in order to reach a great result by reaching the fastest way to send orders to the phone without relying on the usual method used for Mobile Automation, r and it took me very amazing and fast results

1 Like

@feras_assaf

Hello Feras,
thanks for your post.

Yes, all commands work.

Add in your case the path to your Android Debug Bridge, e.g. like this:

RunWait(@ComSpec & " /c ""C:\Users\Feras\AppData\Local\Android\android-sdk\platform-tools\adb"" -s 127.0.0.1:" & $port & " shell input tap 445 914", "", @SW_HIDE)

Let me know your results.

Best regards
Stefan

Hello
It works perfectly and I have tried it
I am now trying to make the following scenario, if it is successful, it will be very impressive
What I am trying to do is to verify the image with a condition, and when the condition is met, it moves to the number

What I am trying to reach is a project on the uipath adb system. Fast connection, because the mobile system atomit is very slow and has problems. This is why I am combining the script with uipath commands. I have received very great results.

1 Like

Hello Feras,
sounds great.
Looking forward to see your finally result as picture or video.
Best regards
Stefan

@feras_assaf

Hello Feras,

in my opinion you can use RunWait without @ComSpec:

RunWait("""C:\Users\Feras\AppData\Local\Android\android-sdk\platform-tools\adb"" -s 127.0.0.1:" & $port & " shell input tap 445 914", "", @SW_HIDE)

On this way you can reduce your application stack from

  1. UiPath
  2. AutoIt
  3. cmd
  4. adb

image

to

  1. UiPath
  2. AutoIt
  3. adb

image

On this way you save memory and loading time, which improves your performance.

Let me know your results.

Thanks and best regards
Stefan

This is my experience integrating your topic with communication by adb comand
https://streamable.com/4e2xuc

1 Like

Hello Feras,
thank you very much. Your integration scenario looks very interesting.
Best regards
Stefan

I am now experimenting with checking the image through Uipath and then executing the script through AutoIt. I will try. If this is done, an excellent program will be formed by merging the two systems together, which is very easy as well.

1 Like