I’ve been trying to scrape some data from a website and the scraping part has been working fine, but the load more button doesn’t want to click to scrape the rest of the data.
I’ve tried different browsers, and I’ve also tried setting the max results value to 100 and 0 but no change.
correct me if am wrong page is not loading to the next page ryt?this is the problem ur facing ?
check the selector for next page as well,
copy the selector of 1st page and paste it in notepad and again copy the next page selector and paste into notepad below first selector compare both the selector if any thing difference is there please make it constant
correct me if am not wrong page is not loading to the next page ryt?this is the problem ur facing ?
Kinda, I’ve noticed that it manages to click the load more button the first time, but then never clicks it again, very bizarre.
I’ve checked the selector and it doesn’t change.
More content loads with ajax so now I’m stuck on what to do here.
I tried copying the entire button element from inspect element but all it does is click it once.
When scraping I want it to click the load more button more than once so it loads all the content.
Nevermind, I got it to work
the reason it wasn’t clicking it a second time is because it couldn’t find a pattern so it assumed there was nothing else to get