I’m trying to scrape search results from the advanced version of google patents (patents.google.com/advanced).
I’ve set up a solution that opens a browser, types in a search term and extracts the results into a data table. I used the data scraping wizard.
As I want more than the results on the first page, I’ve tried to get the wizard to recognize the “next” link at the bottom of the page. In this case I used the “>”-symbol, circled here:
When I marked this link in the wizard, the selector for the “NextLinkSelector” ended up as:
<webctrl id='icon' idx='2' tag='IRON-ICON' />
This only works for the first page. The solution does not proceed to page 3 and instead times out.
So I noticed that after the first results page, the structure of the page changes. It now has controls for going back to the previous page, and this link is very similar to the next-links:
However, as I open the selector for the next link and chose “highlight”, it corretly identifies the icon for the next page:
Also, as I understand it, even if there are multiple elements in the page matching the selector, UiPath should still click the first one.
Still, I went down the path of trying to build an more unique selector. I tried both of these:
<webctrl tag='PAPER-ICON-BUTTON' parentid='link' class='style-scope search-paging x-scope paper-icon-button-0' icon='chevron-right' />
<webctrl parentid='icon' tag='path' d='M10 6L8.59 7.41 13.17 12l-4.58 4.59L10 18l6-6z' />
One of them targets the icon class and the other targets the svg-image in it using the “d” attribute which contains the code determining the direction of the arrow. I still have the same results, the data scraping wont proceed to page 3, but when I open the selector and use the highlight function, it seems to correctly find the element (ie the “>”-symbol).
So why wont it click the link?
I tried unchecking the “simulated click” box to use the hardware driver to click, but as I come to page 2 the mouse pointer wont even move, so it seems like UiPath cant find the element, even though it manages to highlight it when editing the selector.
Or is it a case of the selectors still not being unique?
Any help is very much appreciated! Thank you.
nextlink.xaml (8.7 KB)