Robot i have developed is unable to scrape (100 profiles) data from Linkedin, but it is scraping 5 profiles data (not even number of profiles in a full page)

Help
1

Robot i have developed is unable to scrape the (100 profiles) data from Linkedin,
but it is scraping 5 profiles data (not even number of profiles in a full page),
i don’t have any idea about what could be causing this,
neither giving error nor giving full result.
Even i have said yes for multiple page scraping it is unable to go to next page and it is not giving error also.And i have given 100 for max number of results.
See in the below image it didn’t select all people after jyotsna Sinha.

scraping1
scraping11137×593 60.4 KB

Help me in this,
Thanks
Anjan

2

XML or selector is not properly set in this case. can you paste the xml here from ExtractMetadata field?

3

scraping2
scraping21109×674 63.3 KB

This is how i performed Data scraping but still it is giving 5 results instead of 100 :frowning:

4

<extract>
<row exact='1'>
<webctrl tag='li' class='search-result search-result__occluded-item ember-view'/>
<webctrl tag='div' class='search-entity search-result search-result–person search-result–occlusion-enabled ember-view' idx='1'/>
<webctrl tag='div' class='search-result__wrapper' idx='1'/>
<webctrl tag='div' class='search-result__info pt3 pb4 ph0' idx='1'/>
</row>
<column exact='1' name='name' attr='text' name2='URL' attr2='href'>
<webctrl tag='li' class='search-result search-result__occluded-item ember-view'/>
<webctrl tag='div' class='search-entity search-result search-result–person search-result–occlusion-enabled ember-view' idx='1'/>
<webctrl tag='div' class='search-result__wrapper' idx='1'/>
<webctrl tag='div' class='search-result__info pt3 pb4 ph0' idx='1'/>
<webctrl tag='a' class='search-result__result-link ember-view' idx='1'/>
<webctrl tag='h3' class='actor-name-with-distance search-result__title single-line-truncate ember-view' idx='1'/>
<webctrl tag='span' class='name-and-icon' idx='1'/>
<webctrl tag='span' class='name-and-distance' idx='1'/>
<webctrl tag='span' class='name actor-name' idx='1'/>
</column>
<column exact='1' name='Domain' attr='text'>
<webctrl tag='li' class='search-result search-result__occluded-item ember-view'/>
<webctrl tag='div' class='search-entity search-result search-result–person search-result–occlusion-enabled ember-view' idx='1'/>
<webctrl tag='div' class='search-result__wrapper' idx='1'/>
<webctrl tag='div' class='search-result__info pt3 pb4 ph0' idx='1'/>
<webctrl tag='p' class='subline-level-1 t-14 t-black t-normal search-result__truncate' idx='1'/>
<webctrl tag='span' idx='1'/>
</column>
<column exact='1' name='Location' attr='text'>
<webctrl tag='li' class='search-result search-result__occluded-item ember-view'/>
<webctrl tag='div' class='search-entity search-result search-result–person search-result–occlusion-enabled ember-view' idx='1'/>
<webctrl tag='div' class='search-result__wrapper' idx='1'/>
<webctrl tag='div' class='search-result__info pt3 pb4 ph0' idx='1'/>
<webctrl tag='p' class='subline-level-2 t-12 t-black–light t-normal search-result__truncate' idx='1'/>
<webctrl tag='span' idx='1'/>
</column>
</extract>

5

looks fine idk. can you attach the data scrape sequence you created, here?

6

sequence
sequence988×466 30.6 KB

7

Stuck on a similar problem

8

Hello Experts,

Can anyone attach the workflow(xaml file),for further analysis…

Regards,
Kanthesh

9

I have run into a similar problem when using web scraping on LinkedIn. I have figured out the root issue, but unfortunately no solution yet.
It has to do with the selector for the extract structured data. LinkedIn uses a parentid with en “ember” number. This number changes with each reload of the page.
image
It appears the scraping only works when the ember number is correct, if you replace the number “3511” with a * it doesn’t work.

10

@anjanikumar1802
Try changing your selector for the extract structured data, to something higher up in the ui explorer tree.
I was able to solve this in my case by changing it to:

image
image880×393 18.9 KB

11

Hi @KaylaC, could you please let know how you managed to change the selector?
Thanks!

12

Hey @samjoechandy, @KaylaC, @yas6191, @anjanikumar1802,

Check out this issue…