How to automate/select Windows legacy app that uses drawn components as opposite to Windows objects?

Lets go straigth to the sampe - I am trying to record with ‘App/Web Recorder’.

I have Delphi 6 application which uses DevExpress VCL components. E.g. its menu is composed from TdxBarmanager (nonvisual orchastrator, controller), TdxBar (toolbar/menubar) on which TdxBarButton’s sit (i.e. there are several buttons on one bar and each button open its own menu on which further buttons sit of this kind or another). The essential thing is that TdxBar and TdxBarButtons are not Windows API objects, but they are somehow drawn by DevExpress code.

If TdxBar and TdxBarButton’s were Windows API objects, then ‘App/Web Recorder’ would be able to select each separate TdxBarButton instance. But DevExpress draws entire complex of TdxBar+{TdxBarButton’s} and that is why ‘App/Web Recorder’ selectes entire TdxBar or nothing. But this is not what I need.

I need to select separate TdxBarButton, i.e. I should select only part of TdxBar that corresponds to some specific TdxBarButton.

I can imagine that the Studio recorder could act in the following way in this case: it can switch from Windows-Object-recorder into some select-rectangle regime and when I am selecting rectangle that approximately corresponds to the TdxBarButton, then recorder is able to determine that this region (or the largest part of this region) belongs to Windows component corresponding TdxBar and that selected region is positioned relatively to this Windows components and the selected region can also be recognized by some distinct text that may be captured by as picture with embedded text, i.e. OCR can be used.

I am actively studying UiPath Studio and its documentation right now. I don’t know is such selection is possible or not. But currently I don’t know the way how to do this.

Maybe there are even some usable plugins/libraries for Windows applications that are build using older DevExpress editions?

Thanks for any hints and suggestions in advance!

When you are setting the selector press F3 for image mode. You want to identify the object to click based on its image.

1 Like

Thx, it was easy. Is there some tutorial for this?

But I have further problem. I pressed F3 and rectangle appeared and allowed me to select region and choosse the action click. That was recorded and that was fine.

But I need to perform this click in reality, because this click opens popup menu un and should further select and click button-regions there. So - I should switch from the F3-select_rectangle-choose_click recording regime into actual program execution regime (because chose_click in the recording regime does not execute click and does not open popup menu) where I can execute click in reality. Should I stope and restarting recording regime?

I don’t know anything about using the recorder. I never use it. It’s more reliable IMO to code your clicks one by one - especially when you have complicated situations like this.

1 Like

So, I am in the recording regime, I press F3, select toolbar-button-region and select Click event. Then I am pausing recording and going back to the application and execute click in reality and the menu stays open and it remains open when I am going back to get UiPath recorder. But when I am pressing ‘Record’ (resume-record) button, then the menu closes. I would prefer, that menu stays open and then I could get back to my program and press F3 again and select region from the open menu list.

Indeed, I will try to use activities without recording. Thx.

Again, I don’t use recorder. I find it more reliable to code your clicks one by one.

1 Like

But I should somehow capture the picture of the button on the opened menu. So - I can copy/past this picture in the activity. But any capturing activity (full screenshot, starting recording regime etc.) closes the opened menu and there is no way to capture…

OK - Ctrl+Alt+Printscreen closes the menu, but Ctrl+PrintScreen keeps it open. So - I can capture the entire screena and then I can cut the interesting region manually and paste into activity. I will check it.

You don’t copy/paste the picture in the activity. You click Indicate Element then when the selector window opens you press F3 to switch to Image mode. Then you draw a rectangle to indicate the image/element.

1 Like

I am very sorry for asking so much questions, but I am trying to be good citizen, I have read all the relevant info from “UiPath Associate Certificiation Guide” (there was almost nothing for image selection), abot Object Repository from docs, etc.

But I don’t understand how to select regions in popup menu.

I can try just to capture full print-screen of the screen that has opened popup menu and then I have picture and I can ‘indicate-Element’-‘select-region’ in this static picture. That should work. But I am not sure whether will be OK with screen resolutions and I am not sure whether this is the optimal way, that there is no clever way.

But I can not spark any other way - I can open popup menu only when selector tools is not working and when I am switching to the selector tools, then popup menu closes, so, no region to select.

I just wanted to make this clear - should I capture region from full-print-screen or can I find more optimal way of selecting region in popup menu without taking print-screen?

This will end my inquiries for the moment.

You indicate element then press F3 then draw the box directly in the application window. You must do this so it gets the selector for the window where it’s supposed to find the image.

1 Like

I feel that there is treasure hiding, I just don’t get it.

So. I have UiPath.UIAutomationNext.Activities.NClick on my screen and there is button ‘Indicate in App: MYAPP’. I click on this button and then ‘Selection Options’ screen appears. This is special regime. I can draw rectangle and select button on my menu bar. But this is rectangle drawing regime. I can not press click on this button on my menu bar to open the popup menu and to select region in this popup menu.

So, I tried to press ‘Pause’ and then this rectangle drawing menu exits for some 5 seconds and I can click the button on the menu bar and open popup menu. After 5 seconds the pause regime ends and and the entire popup menu is outlined with green rectangle. But my popup menu contains several entries and wanto to click on one entry only, so I am pressing F3 and trying to select this region, but at this time popup ceases to exist and there is no target to select.

I am not sure.

Maybe my application (it is legacy application) has unconventional behavior. Maybe I am not able to communicate my issue clearly.

Maybe there is possibility to conact some UiPath consultant and have TeamViewer consultation - for just 10 minutes. Currently this use case if blocking adoption of RPA for our application.

But OK. In any case I am seeing some solution - via screenshot. But I still hope that there can be more optimal selection.

Add Click Activity
Click Indicate Element
Press F3
Draw box around button
(If necessary. move bullseye to control exactly where it clicks)

That’s it. That’s all you have to do.

Up to now the disappearance of the menu popup region was my problem why I could not draw rectangle and select region. I discovered ‘F6 Hoverable elements select’ feature this morning and this solves my problem with disappearing popup region of the menu. Now F6 regime keeps the popup open and I can select region in the popup region, some popup entry/button. This is fine.

But selection splits into 2 regions each with their own rectangle and indicator:

  1. the main region, which I am selecting, has rectangle/indicator:
    ////I am prevented to put pics, so I am describing them: [Anchor icon with ‘Anchor’ label]

  2. but there also appears one completely unrelated region in some other part of application with rectangle, indicator and connecting line, like (I split this into two images):
    ////I am prevented to put pics, so I am desribing them: [bullseye] at the corner and some kind of [shooting target] in the middle of the rectangle

I am validating my selection with F7, of course, and then both regions become green and the workflow is without errors. But the execution of workflow misses clicks in the popup region.

What is this - any keywords, terms for this? Can I read more about this in some documentation? But, most importantly, how can I resolve this and move ahead?