Data Scraping activity is pulling URL and Data of only first Page, not of subsequent pages

Help
,
21

Hi @Alok_Shrivastava,
I have worked on this type of project in past. i have observed that click on next button its data is not getting scrape. please go through the attached project,that might be help you.
righmove_IE.zip (35.6 KB)

Cheers

1 Like
22

This seems to be because of the page load. The property cards keep loading even after the page load is complete. That is the reason why extract structured data activity doesnt work fine with this.

Hope the workflow @Jyotika_Halai shared worked for you.

23

Same rule applies for first page also, Why it is working on first page only?

24

Here is how the extract structured data waits for a web page. Got it from the documentation.

  • Web applications:
    a. Internet Explorer - The <webctrl> tag is used to check if the Ready state of the HTML document is set to Complete . Additionally, the Busy state has to be set to “False”.
    b. Others - The <webctrl> tag is used to check if the Ready state of the HTML document is Complete .

You might want to dig deeper into the way the application behaves given these cases, to understand the problem with the extraction.

25

Was there ever a solution found to this problem? I am encountering the same thing as @Alok_Shrivastava + having another data scrape that pulls all of the data except for the first page. I have tried to look around and try different approaches but with no success so far.

26

104
1041363×569 50.6 KB

I am also facing the issue of URL (website is: https://www.jaycar.com.au/)
I attached the screenshot also.
Any idea how we resolove this?
1221
12211323×564 33.8 KB