How to be able to click one digit represented as image in WebPage?


How to be able to click one digit represented as image in Web-Page?
Please find an example below where the positions of digits from 0 to 9 is changing each time.

Note : Digits here are not recognized as button as shown in google online calculator :…0…gws-wiz.OuMXnUMoVSc&ved=0ahUKEwiAtcieq9XjAhXuxcQBHdU-BO0Q4dUDCAo&uact=5

1 Like

Did computer vision activities worked upon them
Cheers @hsendel

1 Like

I tried : Click Image, Click Text (OCR) but didn’t help as the number’s position is changing for each access ( for security reason).

1 Like

Computer vision buddy …
Cheers @hsendel

Thanks Dear, I was thinking you’re talking about UiPath.UIAutomation.Activities, I will try this and come back to you.Thx

1 Like

Cheers @hsendel

Dear Palaniyappan,
I already installed Computer Vision Package but not able to use it, i don’t know if it needs a license or it’s another issue !!!???


Hi @hsendel

For this you dont need license o

If installed means try to close UiPath Studio and try again

Ashwin S

1 Like

Hi AshwinS2
Problem Fixed, I have confirmed to you in the other Topic :


1 Like

Inspection with UIExplorer gives following selector for number button:

html htmlwindowname=‘15643351526200.2460097307028955’ title=‘Accéder à mes comptes en ligne | BNP Paribas’
webctrl idx=‘6’ parentid=‘secret-nbr-keyboard’ tag='A’

You can use idx info for the differen numbers
idx 1 = Number 2
idx 2 = Number 9
next row: idx 6 = Number 0

So with the help of a dictionary you can define a lookup table
e.g Key is number, Value is index

With this the selector can be constructed within an assign activity and be used for the click activity

1 Like

In fact the error above solved but still my reported issue in this topic not yet solved :roll_eyes:

Hi ppr,
seems this will help in my case, but please can you explain more and how you’re able to get UIExplorer output without sharing the XAML file ? Thanks in advance

Hello @hsendel give a try to below attached xaml…
working fine for me in IE
Main.xaml (25.0 KB)

1 Like

I didn’t get your question in detail so I just summarize my analysis work for you:

  • I opened the Link that you have provided in your first post
  • I opened UiExplorer in Standalone mode: UiPath Studio > Design > UiExplorer
  • With the standard functionality “Indicate Element” pointing to one digit button and analyzed the selector result
  • From there I copied to the post

However lets focus on the solution and get your workflow working. I suggest you to work with selectors and Click activity as it has potential to be more reliable and independend to image recognition.

Let me know your open questions

1 Like

Thanks a lot AkshaySandhu, I have already solved the issue using Computer Vision as proposed by Palaniyappan but still your proposal is very interesting, I will check that too. Thx :slight_smile:


Cheers @hsendel

Hi Again,
I have used your file but it’s not working , Could you please double check? ThxMain1.xaml (29.2 KB)

1 Like

Hey @hsendel kindly check the below xaml…
Main1.xaml (46.7 KB)
In previous xaml I forgot to remove the selector and clipping region…
This one is working fine…

1 Like

Thanks AkshaySandhu
Great!!! So we end up with 2 Methods of doing the same thing :slight_smile:

1 Like