Data Scrapping In loop

I need to do data scrapping. But here we need move the inner and outer page to next.

I was using this approach but it only extract the first page data.
I store the get text output in PageCount variable and I Created a variable Counter to loop through pages.
Each time when data scraping ended the next button is clicked and increase the counter by one

Hey @agathiyanv,

While data scraping the inner page, I hope you have clicked ok to extract from all the pages and highlighted next button in the inner page. Please do check.

Vaishnavi V


Can you show us the webpage URL for Page 1,2,3 out of 18 from the first container the outer container in 1st Page


Show us the 1,2,3 out of 18 from the first container and choose the 2 nd page from the second container and show us the snap of URL’s


I did click the option


Select 2 of 18 and show us the URL?
Select 3 of 18 and show us the URL?

Select 2 of 3 and 2 of 18 show us the URL…
Select 2 of 3 and 3 of 18 show us the URL…


The url is not change while navigating the pages

Hello @vaishnavi_velayutham , Its Good idea. But you scraping Per page data and click next on Total page
Try this Method

  1. Your while activity is good, This loop is use only for click next page
  2. In While_page loop, Drag and drop the another while activity and do same concepts for per_page data scraping.
    2.1 Extract the last page number store as integer PerpageTotal,
    2.2 assign PerpageCounter=1
    2.3 Loop the condition
    2.4 condition is PerpageCounter<= PerpageTotal

    2.5 in loop Assign Per pageCounter=PerpageCounter+1
    2.6 If all page over loop exist and goes to page loop and click next page and loop to per page and as per conditions

It is not a fixed pages
For page 1 of 3 - 1 of 18
For page 2 of 3-1 of 4
for page 3 of 3- 1 of 7

That means page 1 have 18 Pages right? You need to scrap all the data of 18 page from page 1?

I need to do for all pages

@agathiyanv , then you have to use Nested loop in while

Loop the page using while is as it it in you flow is good, But data scraping should done within another wile loop. here loop conditions based on perpage

1 of 3 page loop for click activity , within this you have to place another loop for 1 of 18, same conditions use here based on last page and count, count+1

Refer this flow, you may got some idea


1 Like

what is exit condition of inner while loop?


As same as Page’s Last ypu get using Get text activity, get the text of perpage’s last number and set it as limit