Data Scraping does not work with multiple pages of a web page

Help
1

Hi,

I’m trying to extract a table from a web page using the Data Scraping wizard.

The web page displays 50 items per page.

When I had multiple pages (more than 50 items) with “Next Page” link, the data scraping activity “Extract Structured Data 1” worked:

1

  • Selector:
    webctrl css-selector=‘body>div>div>table>tbody>tr>td>table>tbody>tr>td>table>tbody>tr>td>table’ idx=‘2’ parentid=‘acWrContents’ tableRow=‘10’ tag=‘TABLE’ /

  • Next Link Selector:
    webctrl aaname=‘Next Page’ idx=‘1’ parentid=‘acWrContents’ tag=‘A’ /

However, the “Extract Structured Data 1” didn’t work, when I had less than 50 items with no “Next Page” link. When I started the workflow, uipath opened a web page but didn’t go to For Each loop. It ended the workflow without any errors.

I created another data scraping activity “Extract Structured Data 2”, and it worked when the web page had no “Next Page” link.

2

  • Selector:
    webctrl css-selector=‘body>div>div>table>tbody>tr>td>table>tbody>tr>td>table>tbody>tr>td>table’ idx=‘2’ parentid=‘acWrContents’ tableRow=‘9’ tag=‘TABLE’ /

Can anyone please check this workflow and help me out?

Thanks :slight_smile:

2

Hi @anna100

This situation can be handled by handling the next page link click separately from the data scrape wizard.

You can actually maintain a page counter like variable starting from 1.
Then have an Element Exists activity and check whether the page is available. How do you know the page? is using the counter variable. It contains the page number. So each iteration, we can check whether the page 1 exists, page 2 exists and so on…

If the page exists →
Click on the page number to load the next page
Do the data scraping only for that page and add the data to a datatable
Increment the counter variable and loop the process again from the top
So all this can be added within a loop

3 Likes
3

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.