Tb with pagination using data table activity for scraping

Help Activities
, ,
1

I’m scraping data from a website and using the “Extract Data Table” activity to get the URLs of all posts. I also enabled the “Automatically navigate through pages” option and selected the “Next” button selector.

However, when I run the process, it only extracts data from the first 2 pages and then stops.

Can anyone suggest what I might be missing? Do I need to add any extra activities or change something in the configuration?

This screenshot shows a Vietnamese real estate website where users can search for rental properties in Hanoi. (Captioned by AI)
This screenshot shows a Vietnamese real estate website where users can search for rental properties in Hanoi. (Captioned by AI)1890×857 180 KB

2

@Huy_Hoang_Nguyen,

Check the configuration of Table extraction if there is any limit like take first 100 or 50.

3

There’s a possibility that the selector or the “Next” button has changed on the 2nd page. Please check the “Next” button manually on the 2nd or 3rd page using the Properties panel.

4

Hi @Huy_Hoang_Nguyen
Check below,

The image shows a web automation interface for extracting table data from a search results page on Flipkart, including configuring the "Next" button for multiple pages. (Captioned by AI)
The image shows a web automation interface for extracting table data from a search results page on Flipkart, including configuring the "Next" button for multiple pages. (Captioned by AI)1181×790 153 KB

Also check Properties of -Extract table data.

The image shows a configuration screen for extracting table data in a software application, with options for defining extraction parameters and settings. (Captioned by AI)
The image shows a configuration screen for extracting table data in a software application, with options for defining extraction parameters and settings. (Captioned by AI)931×214 23.8 KB

If you found helpful, feel free to tick as a solution.
Happy Automation

5

Hi. thank for your answer. I set the limit value is 0 but it still only retrieves 2 out of a total of 30 pages. A system provides a selector for pagination, and I chose the page navigation icon.
that is url of website " https://batdongsan.com.vn/cho-thue-can-ho-chung-cu-ha-noi?cIds=651,52,577,51,57,576,50,55,53,59&sortValue=1 ". I hope so u can help me.

6

@Huy_Hoang_Nguyen,

I tried to extract and here is the working code. I tested till 10 pages and it is working for me.

TableExtractionSample.zip (1.1 MB)

7

@Huy_Hoang_Nguyen

Some tables behave differently in that case instead of indicating next ..extract only one oage

Use a loop and use clicks to navigate to next page and continue the loop

Cheers