Hi Guys,
I am scraping data from a structured table 3000 rows divided in 30 pages
it takes me 6 minutes to scrap all that, its way too long, how can I make the process much faster ?
One thing I noticed also is that when the 30 pages are loaded once I don’t need internet anymore to switch between pages the data stays available, any idea how I could scrap the 30 pages in one shot in that case ?
Many thanks
If all the data are in one page after the another then it has to navigate across pages
May be what I feel for time taken is because of page getting loaded for few seconds but not more than a min
Being 30 pages I feel it’s fine with data extraction for 6 mins
If you want to try with other options like even if we try with find children it’s going to be the same way to extract which will eventually take same time
Only way we can make it very faster is API
If you have API set up for that web application then we can fetch the same set of data through api in just few seconds
But for now 6 mins is fine with data extraction of 30 pages if it is very stable
**And main thing I want to you check is you are running in normal mode and not in debug mode
When you run your project debug mode it will run in slow pace only
Make sure you are clicking on RUN PROJECT option under that play button**
Hope this would help you
Cheers @kaiji_San