Unable to find stable next link selector - Data scraping multiple pages

Help
,
1

I’m trying to data scrape a site, and having trouble finding a stable next link selector.

This is the site: Bolig til salgs | FINN Eiendom

With some attempts I get the results from the first two sites. At one occation the robot went from page 1 to 2 to 1 and so one until I force quit it.

html app=‘chrome.exe’ title=‘Fredrikstad, Eiendom’ />
webctrl parentid=‘page-results’ tag=‘polyline’ />

This is the selector that alternates between first and second page. I can not reproduce the one that goes from first to second and then terminates.

2

Hello,

Please make sure you select the next page button properly. Try selecting the surrounding button, not the arrow. With this selector it worked for me:

<webctrl aaname='Neste *' parentid='page-results' tag='A' />

You can change the value of the aaname attribute to fit your needs. Hope this works.

3

I have tried that selector as well. That got me from page 1 to 2, but I had
no success modifying that selector to work further (page 3, 4 etc). There
are three different areas to select the button, and I have tried all three
recent to posting om forum.

4

I’m sorry, you are right, the selector I posted works only for the first 2 pages. Paste this into NextLinkSelector property of ExtractData activity:

<webctrl css-selector='body&gt;div&gt;div&gt;div&gt;div&gt;section&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;span' parentid='page-results' tag='SPAN' idx='4' /> <webctrl parentid='page-results' tag='A' />

What I did was to open UiExplorer, indicate the next page button on screen, then from the Selector Editor I checked the closest ancestor of the next page button selector. I also removed the aaname attribute as it was too specific. Also, please make sure to set the MaxNumberOfResults property in ExtractData activity to a greater number, because the default is 100 and that’s only good enough for the first 2 pages.

If the above selector does not work, try different combinations of ancestors/attributes in UiExplorer. Please let me know if it worked.

3 Likes
5

It worked like a charm! I need to work on my understanding of HTML to be able to find those selectors myself. Thank you very much for finding a solution and also learning me something new! :slight_smile: