How to deal with table extraction pagination issue?

Help Studio
1

I am extracting data from a website, it has multiple pages sometimes I only want to extract data from only 3 pages, and sometimes from more…
Sometimes I want to extract from multiple pages but there is only one page available.
How should I tackle this situation?
I am using ScreenScraping > Table Extraction wizard…

image

2

Hi,

you can create a variable to hold the max number of pages to extract, and then you can change the value at runtime based on the needs:

image

Regarding your needs:

CASE 1: I only want to extract data from only 3 pages

In the case you’ll set the value of the variable to 3

CASE 2: and sometimes from more…

In this case you’ll set the variable to a very large number that’s surely greater than the available pages, so you’ll scrap them all.

CASE 3: I want to extract from multiple pages but there is only one page available

In this case you’ll set the value to a very large number, and the engine will detect that there’s only one page available (most likely with no “next” button) so you won’t have to handle the case specifically (basically for case 2 and 3 the setup is the same)

1 Like
3

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.