How to check information inside a application

Help
,
1

Hello people!. I am new to this amazing technology. I am currently working on a robot that will handles invoices. I have a application where some data its stored, a pdf where some information its stored and a robot that will cross check, perfrom actions etc.

Currently in the picture that you see in attachments here, Its a overview of the app that contains some information that i need to crosscheck with my pdf ( i have managed to extract the correct information from the pdf file into variables that i will use to crosscheck) My questions is, How can i extract certaint data and crosscheck it. I have used get text on some of the properties and its working, but some of the elements in the app has yes or no checkbox, some has checkboxes, some has inputfields etc.

I need following, to check every elements in the app, and cross check it with my current variables.

see attachment for better understanding

Thanks in advanced

DBSinforation
DBSinforation1890Ă—649 130 KB

2

Hi @langsem,
You could try to work with Computer Vision activities. Those can make a miracles :wink:
image

2 Likes
3

What activity would be the best approach here ? Find Cv element ?

4

Specialy wonder about those checkboxes, how to identify if the checkbox its checked or not

5

You can use Image exist activity to to one of the checkbox type. Then keep it’s output as a variable (Boolean) and use If activity to determine if it’s checked or unchecked.

6

OK thanks, btw how can i configure the goolge OCR step by step? People tells me that its better to use as its more precise

7

If you mean Google Cloud Vision OCR then here is a quick overview:

For Studio activity you need to provide API key which will work as a kind of “bridge” between activity and your Cloud Vision.

8

The image exist did not work :confused: it could not tell the difference if the checkbox was selected or not

9

Can you show the screenshot of these checkboxes?

10

checkbox1 appwindow

the image at bottom its from the app window

11

Please use only image. Like this:

image
image512Ă—600 13.3 KB

But you need to know about potential problem. This activity will work for any element which will look like this. So if you will have many of these unchecked/checked boxes it will return condition applicable to any one of these visible on the page.

12

Well its alot of this checkmarks. But is it possible to use the computer vision activity get text, then anchor the image to the text ?

13

With CV it’s even possible to use Click CV activity. It recognizes the image as radio button without any problem:
image

14

How will it be performed ? Because those checkboxes can not be checked or unchecked. Its no interaction with them, they are only there for controll state. I have a pdf that contains information and i want to cross check it with the app window. So the robot needs to controll of the checkbox its checked then take the output as variable and assign it to a if statement where it handles rest( but dont do anything with the checkbox just retreive the information based on checked or not checked)

15

Oh… Thought that you wanted to mark checkboxes. In that way it will be hard to approach this.
You could try to experiment with CV Get Text and use anchor with use of Find CV Element:

image
image697Ă—241 7.77 KB

image

16

Did find the element, but dont know how to perform a action to see if its checked or not

17

Hi @langsem

Could you share some selectors of the application? It would help judge the possible solutions :slight_smile:

18

Mean the DOM of the application ?

19

I would like to see the tree from UiExplorer :slight_smile: If there are any, it is possible that there is an attribute that could be retrieved with a Get Attribute activity.

20

Oh i see, yes there its i think just a momen