Using Extract Table Data to scrap data from website without next button

Help StudioX
1

Hi Team,

I am using Extract Table data to fetch data from a website, but the website has no next button. Is there a way I can still scrap data from all the pages? I am pretty new to UiPath and I am using StudioX. TIA!

image

2

@Raghav_Dhawan

you can check if the url is including the page number…then we can use a loop and navigate to each page by changing the page number and then extract the data

cheers

3

Thank you for the suggestion @Anil_G, the URL is not using a page number, but there is a key that is getting added from page 2 onwards and is incrementing by 25. Can this be used? and if yes can you please help how? I am pretty new to UiPath. TIA! Example of URL endings for
PAGE 1
image
PAGE 2
image
PAGE 3
image

4

@Raghav_Dhawan

This is how the floe looks

image
image

in for loop - Enumerable.Range(1,40).ToArray and change the type argument in properties to Integer/int32

in the use application/Browser - url field - “url” + Cint(25*currentitem).ToString
Make sure to give selector of the page…
Then select Ifnotopen for open option
image

If condition is to give a structure to the finaldt which contains all the merged data

dt is the data where extract datatable gets the data and then merges the data in finaldt

Hope this helps

cheers

5

Hi Raghav,
For given problem statement , you can used below steps.
1.Used Loop
2.Create Custom URL
3.Navigate URL
4.Used Extract Data table
5.write append activity
6.clear Data table.

6

Thanks a lot for this detailed flow @Anil_G. I have one quick questions -
the below condition, is being used to add 25 at the end of the url right?
“in the use application/Browser - url field - “url” + Cint(25*currentitem).ToString”

If yes, actually the first page has no common key, and from page 2 onwards the “&start=” + 25 (increment needs to be added). The first pages need to be navigated via the base original URL

7

@Raghav_Dhawan

Can you check if you give start=0 if it is working then just change the enumerable from 1.40 to 0,40

else chaneg the enumerable still to 0,40

and use "url" + If(Currentitem=0,"","&start=" + Cint(25*currentitem).ToString)

and make sure to change the second if condition as currentitem = 0

cheers

8

@Anil_G what if the URL is not including page number? I have a website from which I want to extract data. The page URL is static. Any Suggestions?
The image shows a pagination bar with a highlighted first page, options for the next few pages, and an option to jump to the 92nd page. (Captioned by AI)

9

@shiva.p

Get the selector of that bottom Pages click and pass the number as variable in selector

Cheers