I am stuck on an issue where I need help navigating or clicking on the next page button.
I have 2 webpages with search results. The first webpage has 10 items and the second 8 items.
From the first page I only want to process 3 of the items and get to the next page to process 4 more items. I used data scraping to extract the data to a datatable and am able to process the three required items on the first page. How can I navigate to the second page?
Then indicate that arrow button to navigate to the next page in Data Scraping. After scrapped the data it will ask for you want to scrape all pages then click yes to indicate navigate button. With this you indicate that arrow button.
Why would you do that? Divide the load. Have one workflow that retrieves the URL of each entry. Store said URL in a transaction. Use an workflow to do the heavy lifting, i.e. visiting each URL, extracting and manipulating data, and so on (alternatively, store the entire row’s result in the transaction instead of an URL).
You can do all of this in a single workflow on a single robot, but it’s sub-optimal. Using queues and transactions allows you to split the load between more than just a single robot.