Data Scraping for Webpage - Non visible Data


I am extracting a table from Webpage and only visible data is getting extracting, like for example if there are total 100 rows, and only 10 rows are visible at a time, then only those 10 rows are getting extracted.

Is there any way or settings which can help me extract all the available data and not only which is visible?

Hi @Shreya_Rao1

What about other rows ,are they hidden or it is in other pages?

HI @Shreya_Rao1

The data is appeared on the next page or you need to scroll down to get the data ah?



If your target system is web, can you try to scroll down to the bottom of the table in advance, then try scraping?


I need to scroll down, there is no other way to navigate

Its on same page but hidden , so to check on those I need to scroll down

Still it will give me only those rows which are visible, the above rows will skipped

HI @Shreya_Rao1

Can you able to get the count of the Data that are available in the web page?

If yes, You can get the count and Use scroll activity and get the data ftom the web portal

1 Like

Yes, i am doing same thing currently, but it takes time to do so

HI @Shreya_Rao1

I don’t think so we have some other way to achive this.