How to data scrape multiple pages that need to load by scrolling down each page?

Help Activities
,
1

I tried using the data scraping feature but it’s not enough to do the scraping because, it only gets the data for the ones that has been loaded on the page.

So I took a different path. I used these:

  1. Sequence
    a. Click (Somewhere on the page)
    b. Send Hotkey (end) – this is to scroll down to the bottom of the page so it loads everything.
    c. Data Scrape (without data spanning)
    d. Element Exist (named the variable as Check)
  2. While (condition is Check = True)
    a. Click (next button on the page)
    b. Delay (10 seconds)
    c. Data Scrape (copied the data scraping from the previous)
    d. Element Exists
  3. Write Range
  4. Close Application

It wouldn’t reach the write range activity because it will try to click the last next button. However, the last next button is unclickable or greyed out. So it’s stuck in there.

Zillow Sample Page

2

check with element exists Activity if next button is available and click
check with element exits Activity if next button is greyed out and recognize that paging is done

3

Where do I find it though? Sorry.

4

grafik
Ensure following, maybe toggle to classic
grafik

5

Thanks for the screenshot.
I have already unchecked the “Show Modern”. So I should be in classic now.

Where do I find the option that you previous mentioned under the properties tab of Element Exists activity?

EE
EE1129×498 22.6 KB

Thank you.

6

didnt get you what are looking for?

7

Blockquote
check with element exists Activity if next button is available and click
check with element exits Activity if next button is greyed out and recognize that paging is done

I am not sure how to do the above

8

Hi @jjtubon ,

Try to use AlterIfDisabled property, just make it true it will click button if it is in disabled state.

image
image908×463 21 KB

9

With element exits we get returned a boolean, indicating if the elements was found (using the configured selector ) or not. So you have to configure similar as click or type into.

In case of more knowledge on essential is needed go to UiPath Academy and check out for some training courses.

1 Like
10

Thanks for your response. Well, I don’t want the click activity to click the greyed out button. If the next page button is greyed out, I just want to end the everything and save the excel file with the write range.

11

I have realized that the problem here is in the Element Exists. I created a variable for ELement Exists named “Check”. I used the condition on my while loop as if Check = True. So meaning, as long as Element Exists can find that button, it will continue looping. However, the problem here is that the button is there but it’s no longer clickable because it’s the last page. How do I stop this loop and just proceed to write range?