Web data scrapping if Next link is not available

scraping
datascraping

#1

Hi, I have a situation where i need to extract data from a web page and found Next button is not present instead simply displaying page numbers. Any idea how to overcome as the page numbers are dynamic based on the entity we select for scrapping data.


#2

Hi,
As you mentioned its just a number no hyperlink to it then how your navigating to next the page ?
Did you check this.


#3

hi,
The hyper links were present with the page numbers itself but No Next button to tell the robot to move on. See exactly as below:


#4

Hi,

Could you click on 2 when data scrape prompt for the next link and paste the extractmetadata property of of data scraper here.

PS:.you just need to pass the counter variable in page number of metadata.


#5

extractmetadata property is

NextLinkSelector is this:


#6


#7

Hi,
Can you change column name by 2,3, and so on and check whether scraper giving different output.


#8

hi, i tried this and it is just scraping only data displayed on first page itself like when get_columns_name=1.
But i come over this by writing a decision block, calculate the number of pages and loop over to move on to the subsequent pages.


#9

I was trying to see how to pass variables to extractmetadata property and figured it out myself. Just pass the variable in single quotes.