I am stuck on an issue where I need help navigating or clicking on the next page button.
I have 2 webpages with search results. The first webpage has 10 items and the second 8 items.
From the first page I only want to process 3 of the items and get to the next page to process 4 more items. I used data scraping to extract the data to a datatable and am able to process the three required items on the first page. How can I navigate to the second page?
Then indicate that arrow button to navigate to the next page in Data Scraping. After scrapped the data it will ask for you want to scrape all pages then click yes to indicate navigate button. With this you indicate that arrow button.
I am able to do data scraping completely fine and get the output in a data table.
Steps that I have to follow:
Scrape data in a data table. (total 18 rows)
Come back to first page and click 3 items (out of 10) on the first page and do some processing.
Once this is done I have to go to the second page and process 4 (out of 8) items from the page.
The issue like I mentioned is navigating to the next page. I am able to solve it using counters and when the count reaches 10 it will click on the next page button.
Why would you do that? Divide the load. Have one workflow that retrieves the URL of each entry. Store said URL in a transaction. Use an workflow to do the heavy lifting, i.e. visiting each URL, extracting and manipulating data, and so on (alternatively, store the entire row’s result in the transaction instead of an URL).
You can do all of this in a single workflow on a single robot, but it’s sub-optimal. Using queues and transactions allows you to split the load between more than just a single robot.
There are 10 web pages and I have to navigate to each and every page and I have to extract data into a csv, but there no next button to scrap the data, please help me.