How to get a dynamic selector for data scraping multiple pages

Learning Hub
1

Hi,
so im trying to scrape data from multiple pages. I already figured out how to read all the pages but the problem is that the process goes through all the pages but it only writes the first web page and not all of them. I’m trying to write the data in excel. I scraped data from the first page and the 5th and i compared the two selectos and they are both the same. Does anyone know what the problem might be.

This is the selector of the data scrape.

Thanks

2

Zajeddta slika

3

@nejc_bicek,

Please go through below link and follow carefully 13 step…!!, I think you stuck there.

Please let’s know if you stuck anywhere :slight_smile:

Cheers,
Pankaj

4

The problem is that this page doesn’t have a “next button” it only has an arrow that if you click it, it takes you to the last page and not to the second one.

5

Do you have any difference in the URL of the pages?

7

www.something.com/abc/def/1
www.something.com/abc/def/2
www.something.com/abc/def/3

It changes like this

1 Like
8

@nejc_bicek Just use navigate url activity and pass “www.something.com/abc/def/” +counter variable

9

How exactly do i do that, I’m fairly new to this so i’m still figuring things out.

10

If you know the URL pattern, won’t it be possible to navigate to the pages without relying on the onscreen button?

11

I’m sorry but I don’t understand what you are trying to say.

12

Hi @nejc_bicek

Assume that you have 10 pages for scrapping the data which are of fixed format and have same structure.

Steps :-

  1. Open Browser with the URL “www.something.com/abc/def/1”
  2. Assign a variable as int32 value as 2 initially
    image
  3. Use While Loop Activity as below
    image

Mark as solution and like it :slight_smile:

Happy Automation :raised_hands:

Best Regards
Er Pratik Wavhal :robot::man_technologist:t4: :computer:

1 Like
13

In the navigate to sequence there is an error it says that it cant be converted from string to double.

14

Hi @nejc_bicek

Check the variable you have taken in assign activity is of Type Int32 as shown in my 2nd step.

Happy Automation :raised_hands:

Best Regards
Er Pratik Wavhal :robot::man_technologist:t4: :computer:

15

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.