DATA SCRAPING NOT WORKING BECAUSE NEXT PAGE BUTTON NOT VISIBLE

Help Something Else
1

Hi. Can someone help me? How to fix the problem where bot unable to scrape data as the next page button is not visible?

2

how do you do it manually ? Must you do something to make the next page button visible?

3

should be scrolling only. but i don’t know how to scroll while scraping.

4

@nr.arinaazizi

May I know how many number of rows it is scraping if you are not indicating any next page button ?

5

I should get 100 rows. The first page only have 40+

6

@nr.arinaazizi

I guess you set MaxNumberOfResults to 100 because of that you are getting 100 rows. If you want to scrape all rows then set to 0.

7

image
what should i do to get pass through this?

i checked the properties and it’s already string.
image

i have a problem in this part

image
image603×551 30.1 KB

8

@nr.arinaazizi

Can you please share the screenshot of the Extract DataTable properties.

9

image
image742×690 27 KB

here is it

i set continueonerror to false so i can catch this issue or else my datatable is empty

10

@nr.arinaazizi

Here you don’t have any Next page navigator right ? If so you can delete Next Link selector and then try once.

Are getting this error when you changed set the MaxNumberOfResults to 0 instead of 100 ?

11

Here I have next page navigator and able to get through until next page and able to scrape the data until 46 lines. No, I get the error even before I changed it to 0. It’s kind of hit or miss.

12

@nr.arinaazizi

Can I access this website from my end ? If yes then can you please share me the URL to me. I will check and let you know.

13

yup, it’s just https://shopee.com.my/

14

it looks like i only getting the error after set the MaxNumberOfResults to 100. had to rerun few times to make sure of it.

15

@nr.arinaazizi

I searched for bag and able to scrape data like Name, URL and Price from all pages. Please find the attached workflow for your reference.

Shopee_DataScraping.zip (1.1 MB)

16

still didn’t work for me but thank you. your effort is appreciated.

17

@nr.arinaazizi

Still getting same error ?

May I know your webpage is in which language ?

18

Hi @nr.arinaazizi ,

Please use the below workflow. It worked perfectly for me. You can also modify the same.
Hope it helps.

DataScrapping.zip (5.2 MB)

19

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.