DataScrapping on an infinite scrolling page

[Constance_Termoz]

Hello,

I am new to UiPath and just getting started:
I would like to scrap data on an infinite scrolling page (the page keeps loading for new results when scrolling down to bottom of page).
Unfortunatly, the only answer I found about this topic is this one:

Data Scraping doesn't work due to new results loading when scrolling to bottom of page

Hi, so here is the issue: I am trying to extract all links related to a certain search. The problem is that only a couple of results are displayed when the results list is loaded, and that additional search results are only loaded when you scroll down to the bottom of the page. So I do not have the option to indicate a next button or something of the sort to tell UIPath where it can find the next batch of links to look for and scrape. Do you have any idea how I could solve this so that UiPath…

Please can somebody help me with a screenshot of a scenario ?

Thanks by advance,

[ppr (Peter Preuss)]

@Constance_Termoz
welcome to the forum

in such scenarios following strategy was often helping:

• simulate the scrolling with pg-down key or hovering to a particular element (e.g. a load more button)
• identify an information that indicates that it is scrolled to end (e.g. no more load more button or 356 of 356 items statistic line)

once it is cleared then in a while loop the scrolling can be implemented till it is scrolled to the end. Variations on this patternis also often possible (e.g. extract data, scroll… etc.)

Maybe you can share the url with us

[Constance_Termoz]

Hey,

Thank you for your quick answer,

Well, I want to scrap data about members in a Workplace group and unfortunatly because it is a private page I cannot share an url

Taking your advice into account, I managed to do this scenario but I am stuck there at the condition step:

• I know the result of my DoWhile loop should be boolean (True or False)
• But I do not know how to name this in the purpose to write the condition
• Or if there is another way to create a condition like “When you do not see this image, stop scrolling down”

Thank you

[ppr (Peter Preuss)]

lets assume the image exists is triggering with its output result e.g. the bool variable isFound if the scrolling should continue or not. then just use isFound (or your declared variablename) within the dowhile condition

[Constance_Termoz]

Hello !

So I find a solution, which needs improvement maybe but works for now and I share with this community in case of:

Because the scrolling downloading logo was to fast I did not success to capture the image so instead of using a DO WHILE loop I used a simple WHILE loop
Also, I used a type into and not a send hotkey because the keyboard shortcut for scrolling down of my computer is fn (lctrl) +end

Do not hesitate if you have any questions

Constance