How to be able to click one digit represented as image in WebPage?

Help
1

Dears,

How to be able to click one digit represented as image in Web-Page?
Please find an example below where the positions of digits from 0 to 9 is changing each time.

https://mabanque.bnpparibas/fr/connexion

image
image430×593 37.6 KB

Note : Digits here are not recognized as button as shown in google online calculator :
https://www.google.com/search?ei=aGM8XYDhAe6Lk74P1f2Q6A4&q=online+calculator&oq=online+calculator&gs_l=psy-ab.3..0i71l8.0.0..460802...0.0..0.0.0.......0......gws-wiz.OuMXnUMoVSc&ved=0ahUKEwiAtcieq9XjAhXuxcQBHdU-BO0Q4dUDCAo&uact=5

1 Like
2

Did computer vision activities worked upon them
Cheers @hsendel

1 Like
3

I tried : Click Image, Click Text (OCR) but didn’t help as the number’s position is changing for each access ( for security reason).

1 Like
4

Computer vision buddy …
Cheers @hsendel

5

Thanks Dear, I was thinking you’re talking about UiPath.UIAutomation.Activities, I will try this and come back to you.Thx

1 Like
6

Sure
Cheers @hsendel

7

Dear Palaniyappan,
I already installed Computer Vision Package but not able to use it, i don’t know if it needs a license or it’s another issue !!!???

image

8

Hi @hsendel

For this you dont need license o

If installed means try to close UiPath Studio and try again

Thanks
Ashwin S

1 Like
9

Hi AshwinS2
Problem Fixed, I have confirmed to you in the other Topic :

10

great

1 Like
11

Inspection with UIExplorer gives following selector for number button:

html htmlwindowname=‘15643351526200.2460097307028955’ title=‘Accéder à mes comptes en ligne | BNP Paribas’
webctrl idx=‘6’ parentid=‘secret-nbr-keyboard’ tag=‘A’

You can use idx info for the differen numbers
idx 1 = Number 2
idx 2 = Number 9
next row: idx 6 = Number 0

So with the help of a dictionary you can define a lookup table
e.g Key is number, Value is index

With this the selector can be constructed within an assign activity and be used for the click activity

1 Like
12

In fact the error above solved but still my reported issue in this topic not yet solved :roll_eyes:

13

Hi ppr,
seems this will help in my case, but please can you explain more and how you’re able to get UIExplorer output without sharing the XAML file ? Thanks in advance

14

Hello @hsendel give a try to below attached xaml…
working fine for me in IE
Main.xaml (25.0 KB)

1 Like
15

@hsendel
I didn’t get your question in detail so I just summarize my analysis work for you:

  • I opened the Link that you have provided in your first post
  • I opened UiExplorer in Standalone mode: UiPath Studio > Design > UiExplorer
  • With the standard functionality “Indicate Element” pointing to one digit button and analyzed the selector result
  • From there I copied to the post

However lets focus on the solution and get your workflow working. I suggest you to work with selectors and Click activity as it has potential to be more reliable and independend to image recognition.

Let me know your open questions

1 Like
16

Thanks a lot AkshaySandhu, I have already solved the issue using Computer Vision as proposed by Palaniyappan but still your proposal is very interesting, I will check that too. Thx :slight_smile:

2 Likes
17

Fantastic
Cheers @hsendel

18

Hi Again,
I have used your file but it’s not working , Could you please double check? ThxMain1.xaml (29.2 KB)

1 Like
19

Hey @hsendel kindly check the below xaml…
Main1.xaml (46.7 KB)
In previous xaml I forgot to remove the selector and clipping region…
This one is working fine…

1 Like
20

Thanks AkshaySandhu
Great!!! So we end up with 2 Methods of doing the same thing :slight_smile:

1 Like