"undefined" error data scraping

What does it mean when you select the 2nd item in the scraping wizard and it simply tells you “undefined error” and wont let you select the item?

That states you are not selecting the similar elements or tags while selecting the second element @ColinCrabtree

please check if you are selecting the same tags by opening the UiExplorer

It’s tricky sometimes. The second element might be an <li> or <div> element inside a <td> element.
Hovering the pointer over and inside the second element might output the right selector.

@AndyMenon Tricky it is! I had a look with UI explorer and found a selector that may work. However I need to make it dynamic. I need the idx to be incremented. What is the correct way of doing this. The highlighted area is not working. The way its coded here i found elsewhere on the forum but seems incorrect.

“<webctrl css-selector=‘body>div>div>div>div>section>div’ idx=’"&SubCatIDX&"’ parentid=‘bonDiv’ tag=‘DIV’ />”

You could convert this into a VB Expression and replace the screen-pointed selector with the VB expression .

I think you can remove the entire css part as it’s bound to cause the selector to fail if the page style changes. The parentid could be wild carded to make sure that the selector won’t fail if it’s moved under a different parent in the future.

The VB expression would look something like a string.

“<Webctrl Idx = ‘”+ __SubCatIDX.tostring() + “’ parentid=’*’__ tag=‘DIV’ />”

Cool ! Let me try this. :slight_smile:

Hi @AndyMenon. Excuse my ignorance here. I am not to clued up with VB. Do the “_” underscores have some significant meaning ?

Nope, looks like they got in there when I formatted that text . Please remove them.

Here’s one that should be syntactically correct. Looks better than the one I was typing from my phone :slight_smile:

"<Webctrl Idx = '" + SubCatIDX.tostring() + "' parentid='*' tag='DIV' />"

Awesome Andy. will give it a bash !! :smile:

@AndyMenon The dynamic selector worked for one index but not for others. I tried some stuff with the UI Explorer. The image below shows the area I wan to scrape as highlighted by the Explorer

This is the explorer image showing selector. I copied and pasted it to the data scrape activity with no dynamic options at this stage and it produced nothing. Just a blank table.

I don’t understand why if the UI explorer is indicating the data I want to scrape with a selector, that the selector does not work in the Data scrape activity.

Ok, that Idx value needs to be removed.

Also, that css part also needs to go.

On the UIExplorer, there is a list of “Unselected Items”. Just because it isn’t automatically selected by UiPath doesn’t imply we can’t use them.

If you expand that box, what is the list of unselected items you see?

This is full expanded list

Can you give this a shot? If you can parameterize the aaname with the name of the plan instead of an idx value, it should be work.

image

Did this work?

Hi Andy. No I am struggling. I did as you said above.
The below image shows the sub-section I am trying to extract from the indicated web page. There are 6 such subsections.

This is the selector I use. The aaname seems ridiculously long. In this example I have not even used a dynamic selector yet. I am just trying to get the extract working as is first.

The results are shown below- an empty data table.

Hi Colin,

In the first screen grab, the ridiculously long selector is probably because the web scraper does not resolve that grid to be a table. That selector will not work in the long run if the items on the list changes.

Can you try pointing to the column that says HOSPITAL BENEFITS, which is the header row for that table and try again?

Hi Andy

I have tried every selector under the sun using UIExplorer. The only set that produces anything is this one:

This is the highlighted area:

This is what it produces. It extracts the data from the second tab “Hospital Benifits” only and comma delimits each piece of data on a separate line.

I think i must give up on this one. I don’t think this page is a good candidate for automation and scraping.

Colin, is this structured data scraping you’re trying? If yes, then Sometimes the processes are limited by the native design of the application or page we are trying to automate.

If this isn’t structured web scraping, did you give that a shot?

I think now that you’ve gotten this far, you can consume this array and loop through it and process the data. If I’ve understood right, the first element of the array is the current year (2019). After that there seems to be a pattern in the array where the first item is the service, and the second item is the price .

You could read those values and add those to a Datatable.

Thanks.

Hi Andy

Yes, this is structured scraping. The weird thing is the array it created was only for the 2nd tab “Hospital benefits”. No matter what I do I cant get a selector to scrape the other 5 arrays… will keep at it as a learning exercise in selectors and try an unstructured approach as well. :smile: