I have access to a SAP BusinessObjects Analysis webpage. I want to use data scraping in order to get the data out as it contains over 300k cells of data and using the export to excel facility only allows a maximum of 100k cells to be exported at one time.
The problem is that the way the page is setup, it only loads 300-400 results at a time and you need to PgDn or scroll 2-3 times in order to load the next lot of results. Due to this when I use the NextLinkSelector to scroll down, it does so once (not triggering any extra data to be loaded) then stops after the timeout period expires.
Is there any way to trigger the NextLinkSelector multiple times? Or have it in a loop where the data is not duplicated?