TUI Scrape data until no more search results

Help
1

Hi All,

I was just wondering if there is a way to scrape all the locations from each place (meaning the dynamic 200+ amount which isn’t loaded as one page)

I understand you can use Page down or end with element exists but If I put element exists for the bottom of the page it will end shortly if the page doesn’t load quick enough.

TUI Holidays webpage
Thanks in advance :slight_smile:

2

Hi Stabbathehut,

Have you tried with data scraping, specifying span in multipages ?

3

But it’s scrolling down. What I currently have is using hotkey end then page up until it finds the element exists that says all matches found but doesnt seem to be doing it

4

Is there any specific button on the screen to scroll down ?
Use on element appear activity with generic search result selectors and have some time in timeout for the page to load, place the sequence to scrape and page down inside the on element appear activity, set the repeat property to true(which is set by default ) and continue on error property as well to true to stop without an abnormal termination.

5

Could you have a look at the link I attached? Its got the generic scroll bar and I have tried adding hotkeys pg down but no luck

6

@Stabbathehut
the Bot could scroll till all is loaded:

End Condition:
grafik

Scrolling:
Hover on the load more dots

Have a look on this preimplementation:
Stabbathehut.xaml (9.2 KB)

I suggest for testing

  • filter the results before to reduce the search results
1 Like
7

Hi there, that’s great!

I was able to get it working via your code. Just wondering if you have any idea in why the do while condition in mine is not accessible. All I did was copy and paste yours over yet I have no luck with this.

image
image1920×1040 91.4 KB

8

you have to adjust the variable scope of isEnd to a higher level.
click on the activity that has output the isEnd variable, and modify the scope within the variable panel

1 Like
9

Also final thing, how did you click the loading button before it disappeared? Sorry to be a pain but all so far really helpful

10

i didnt get your question, can help me with some more descriptions. Thanks

11

The hover on the dots to load more. I can’t change as it doesn’t seem to be picking it up. It loads too quickly to capture the loading dots

12

was not clicked with indicate on screen.
Here an analysis in UiExplorer and FireFox inspect element was done. With the information from this I wrote the selector

grafik
grafik1058×237 9.87 KB

13

Is it something like this in Chrome. I just find Chrome easier to use and more compatible for what I am doing with code. Would it be possible to find what element it is if its not what I put above?

14

chrome is not installed / made some troubles (personal pc issue, not UiPath related) but give a try on F12 or right click on any element in the webpage and look for menu entry like inspect element

15

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.