Automate data scraping multiple pages

Learn Academy Feedback
1

Hi everyone I go to the forum as I tried the web scraping function of ui but I can not solve this problem: on the web page I have several links that correspond to as many pages that each contain a table that I need to acquire and convert to csv . The pages that contain the links are more than one so I would need a feature that will enter the links on each single page and process all the pages with the “next” feature. I hope despite my English to be able to explain. thanks for the help you will give me.

2

Hello,

Let me see if I understand correctly. There is a list of links that has many pages and you want to access each link and extract the data from each link. Here is one approach you can take:

  1. Use Open Browser activity to access the main web page.
  2. Use Data Scraping to extract all links from all pages. More on Data Scraping here and here.
  3. Use For Each Row activity to iterate the DataTable resulted from the previous step.
  4. While iterating:
    4.1. Use Navigate To activity to navigate to each URL.
    4.2. Use Data Scraping to extract the data from the table.
    4.3. Use Write CSV activity to save the DataTable resulted from the previous step in a .csv file.

I hope this points you in the right direction.

1 Like
3

Hi I have been trying to follow these steps above in order to scrape data from multiple pages the problem I get is after the navigate to section. I dont know how to start the data scraping for the individual tables and to write and save the results