Data Scraping with for each loop for multiple links showing UI attribute exception error

Help
1

I am using data scraping using for each loop.it get the data fir the first url but for other url it do not.

I have url stored in my datatable which i am using to get the url.

I think this behaviour is due to selector of data scraping in which title attribute is of first url.

I tried changing the title attribute of selector of data scraping to asterisk(*) but i do not get any data in that case

2

Can you add data scrape activities for two pages individually and share the work flow? I can help comparing what needs to be generalized.

3

@paras_mittal

Why are you using data scraping with For each loop.

Is it structured data ? If yes, then use Data scraping activity and indicate next page navigator to retrieve data from all pages. And the final result will be stored in dataTable.

4

Hi lakshman,
Data need to taken from different web pages and data is structured in website.That’s why i am using data scraping using for each loop.

5

Hi kaderms,
These are two link
https://www.manta.com/mb_34_E229F_000/electron_tubes
https://www.manta.com/mb_34_E22A3_000/electronic_capacitors

I need to know what selector value to use to get “All featured company” data using data scraping.

6

Yes, we have to use data scraping here to scrape data from different pages as it is structured data. And also click on next button to indicate page numbers like 1,2,3 etc…
It will scrape data from all web pages and will store result in dataTable.

7

The webpage structure differs a little bit with reference to the html page. Compare the ExtractMetaData parameter of the Extract Data activity, while using data scrape for each of these pages.
Might have to generalize this for all the pages that you need to extract data from.
I will try sharing for the two pages that you have mentioned above.

8

thanks kaderms.

9

@paras_mittal

I observed the table patterns on both the pages. They seem slightly different. In this case the same meta data would not suffice for extraction from both the tables. not sure about the other pages.

Page 1 - One <ul> element has all the list items within it, under Featured Company Listings.
Here the pattern is straight forward for data extraction

image
image953×402 37.7 KB

Page 2 - There are two <ul> elements here. The first one holds only one result, and the second one holds rest of them. This does not show any difference on the UI displayed though. But this is what affects the extraction pattern for data scraping.

image
image961×432 41.1 KB

10

So how to go around solving this issue -
My best suggestion would be to talk to the developers if you know them and check if this is an unexpected behavior, as two result containers are displayed. If it is abnormal and if they can fix it like page 1, problem solved - same meta data is enough to extract from both the pages and your existing code would work.

The only other way around that I can think of is to observe all the patterns for the pages and separate out the ones which are abnormal like page 2 - have an specific data extraction methods for the abnormal pages. For the normal pages with the pattern like page1, you can loop them through.

Hope this helps

11

Hi @paras_mittal

Can you tell me what exactly are you trying to scrape and do you want to use only data scraping?

Are the two ‘manta’ websites you want to run data scraping upon?

I can do a small POC on that if so.

Regards :slight_smile:

12

In a same webpage, i have a dynamic value, everytime it changes, I’ll have a new DataTable. I need to scrape it each time but, with the data scraping, I’m getting Attach browser/window also.
Everytime that dynamic value changes, the Data Scraping output is same as the data
I’ve used to scrape.