Unable to find stable next link selector - Data scraping multiple pages

I’m trying to data scrape a site, and having trouble finding a stable next link selector.

This is the site: https://www.finn.no/realestate/homes/search.html?location=0.20002&location=1.20002.20024

With some attempts I get the results from the first two sites. At one occation the robot went from page 1 to 2 to 1 and so one until I force quit it.

html app=‘chrome.exe’ title=‘Fredrikstad, Eiendom’ />
webctrl parentid=‘page-results’ tag=‘polyline’ />

This is the selector that alternates between first and second page. I can not reproduce the one that goes from first to second and then terminates.

Hello,

Please make sure you select the next page button properly. Try selecting the surrounding button, not the arrow. With this selector it worked for me:

<webctrl aaname='Neste *' parentid='page-results' tag='A' />

You can change the value of the aaname attribute to fit your needs. Hope this works.

I have tried that selector as well. That got me from page 1 to 2, but I had
no success modifying that selector to work further (page 3, 4 etc). There
are three different areas to select the button, and I have tried all three
recent to posting om forum.

I’m sorry, you are right, the selector I posted works only for the first 2 pages. Paste this into NextLinkSelector property of ExtractData activity:

<webctrl css-selector='body&gt;div&gt;div&gt;div&gt;div&gt;section&gt;div&gt;div&gt;div&gt;div&gt;div&gt;div&gt;span' parentid='page-results' tag='SPAN' idx='4' /> <webctrl parentid='page-results' tag='A' />

What I did was to open UiExplorer, indicate the next page button on screen, then from the Selector Editor I checked the closest ancestor of the next page button selector. I also removed the aaname attribute as it was too specific. Also, please make sure to set the MaxNumberOfResults property in ExtractData activity to a greater number, because the default is 100 and that’s only good enough for the first 2 pages.

If the above selector does not work, try different combinations of ancestors/attributes in UiExplorer. Please let me know if it worked.

2 Likes

It worked like a charm! I need to work on my understanding of HTML to be able to find those selectors myself. Thank you very much for finding a solution and also learning me something new! :slight_smile: