Urgent help needed in scraping data from multiple pages

Off-Topic Discussions
, , ,
1

I am at a page where there are say 50 results I want to go to each results there are name and number I want to scrape both then want to go back to page again.

So the scenario is like say 50 results in one page, so click on 1st result, from there scrape name and number, go back, go to 2nd result again scrape name number go back, and so on for 3rd 4th upto 50.
And at end want to store all scraped name and number in excel sheet.

I tried a lot of scenarios but it is either extracting only 1st result or nothing happening. Please help this is really urgent

2

Duplicated topic

3

Hi @SHIVAMMISHRA6

Can you try this-

  1. Use the “Data Scraping” wizard to scrape the name and number fields from the first result on the page.
  2. Then use the “Click” activity to navigate to the next result on the page. You can use the “Find Element” activity to locate the “Next” button or link on the page.
  3. Then use “Write Range” activity to store the data in an Excel sheet. You can create a new Excel sheet or append the data to an existing sheet.
  4. Use the “Click” activity to navigate back to the original page and repeat steps 2-3 until you have scraped the data from all 50 results.

Thanks!!!

1 Like
4

Hello, thanks for replying.

But this only works if everything is present in one single page.

My scenario is like there is a X page, on that 50 links are there and I want to go in each link, want name and number from there, then go back to X page and again in 2nd 3rd 4th link, like that.

I’m unable to create a sequence for that, where i can scrape the name number go back to X page again click on 2nd link scrape name number, back click 3rd and. So on.

Please suggest as I’m new to this and already tried for 2 days

5

Hi @SHIVAMMISHRA6

To scrape data from multiple pages, you can use a combination of activities such as “Data Scraping”, “For Each Row”, and “Navigate To” in UiPath. Here are the steps you can follow:

  1. Use the Data Scraping wizard to extract the URLs of the links you want to scrape data from and save it to a DataTable.
  2. Use the For Each Row activity to loop through each row in the DataTable.
  3. Use the Navigate To activity to navigate to the URL in the current row.
  4. Use activities such as Get Text or Get Attribute to scrape the required data from the page.
  5. Store the scraped data in a DataTable or any other data structure of your choice.
  6. Use the Navigate Back activity to go back to the original page.
  7. Repeat steps 3-6 for all the rows in the DataTable.

Thanks!!

1 Like
6

Thank You @Nitya1

I will try this method.

1 Like
7

Hello @Nitya1 , It didn’t worked. I’m trying different things from past 2 weeks.

I see you are experienced. Can you please connect with me and help me out that would be great. I need this to be done very urgently.
Please help me out

8

@Nitya1 is there any chance we can connect, a reply would be helpful