Data scraping table. go to next page. scrape that. add contents of tables

Help Studio
, ,
1

Hi

I have sequence that reads in data from a table in a web page, appends that data it to a .csv file.

it is in a do while where the bot checks how many pages are in the data table and goes to next page if there is more data on the next page. the bot then appends that next pages data onto the .csv.

This process works in debug mode when i step through it. However when i run it it only puts the first pages data in or it will put the first page data in twice.

The reason i am not using the Next link selector in the data scraping is because it will press the next page button even if there is no next page

image
image1628×420 35.6 KB

Sequence.xaml (20.1 KB)

Any ideas on how to do this more efficiently?

Ignore the extra data scraping sequence at the bottom of the sezuence

2

For every page check the selector next page button using UIexplorer tool and match with the selector you are using in click next page activity.

3

I set the ContinueOnError to False on the data scrape activity. it failed on the second page even though when it stopped and validated it and highlighted it and it matched

image

4

if possible share the url

5

it is confidential sorry. but the selector for the click next activity is okay. In debug mode it failed on the above again. When i clicked retry on the error and it worked okay. Very strange

6

I think i see what you mean. At the moment i am using a virtual machine and in order to click the next page button i have the selector on class=‘lastPage’ rather than class=‘nextPage’ as they don’t match up correctly.

i ran it on my own PC using class=‘nextPage’ and it worked fine. i will need it to work on the virtual machines though…