Scraping Social Media data

Help
, , , ,
1

Hi All,

First post within the community. I am fairly new to UI Path. I am trying to currently scrape some data from the site LinkedIn via UIPath and output this into Excel. I am searching for any company (does not matter) and then trying to find people (workers) within that company with a specific i.e. assistant as a job title. I want to output their name, job title, company and email if it exists into an excel file.

I have tried to use UIPath to open the browser and go to the specific section but I am unsure on how to output the data with specific headers into a csv or .xlsx

This should be done for anyone who has the keyword assistant in their title. I would like to attach this via email and send it to myself or anyone encrypted.

Any help with this issue would be a massive help.

Many thanks in advance.

2

Did you try using Data Scraping wizard to extract all the data from the search results page? Once extracted you can filter the data table to have only the specific assistant title.

You can then use Send SMTP Mail Message/Send Outlook Mail Message/Send Exchange Mail message as your need to send the email.

Let me know if thats not what you are looking for.

Thanks,
Rammohan B.

3

Hi @Rammohan91 thanks for getting back to me.

I will surely try this out. I am at the moment trying to get past the stage of searching within the intial screen of Linkedin. I have used the recorder and typed a company name. When I try and run the bot it does not seem to type the name of the company I typed (whilst recording). It is very frustrating any idea?

image
image727×188 6.77 KB

I omitted that specifc parentid as I noticed this changes within linkedin. It worked 2 hrs ago but for some reason I cannot get it to work again :frowning:

Any ideas and suggestions would help.

Note: I have created a variable called UserInput via a input dialog that I get “the user” to fill when the robot runs.

Thanks

4

With your selector don’t omit just do parentid=‘ember*’ or parentid=‘*’ or in the edit selector screen click “attach to live element” and select the search field.

1 Like
5

You need to make your selector more dynamic & unique. Try to play around with UiExplorer and look for the attributes that doesn’t change. Something like ‘aaname’ may be.

Note: LinkedIn has a way to detect automation’s on the site and i believe it may block you after few tries while scraping the data.

Thanks,
Rammohan B.

2 Likes
6

@Rammohan91 you are right LinkedIn is trying to stop me in my tracks but I am persistent…partly your solution and @CBlanchard worked a treat. I used a wild card within the ‘aaname’ this allowed me to input user defined data.

Moving forward I am in dire need of some help. I am now at the data scraping part which to be honest works like a charm. My only problem is when the page reloads (so for instance if I refresh the page or I try and run it the next day) I don’t seem to be able to get the datatable to write into a csv or excel file again. When I datascrape it and write range it works the first time but if the webpage has been reloaded (or its the next day) it stops working…

I have tried using wild cards but unfortunately this is not helping this time. I am running out of ideas. The concept works but I cannot get it to output if I reload the page. The metadata has not changed either.

I am at a crossroads, any help on this matter will be highly appreciated. I feel like I am at the verge of finishing the task but I am unable to get this last part done.

image

image

Thanks

7

@xkarrox Hey, any good luck? Im trying to do the same but I am stuck as data scrapping returns nothing…

Please share any solution.

8

Hi, did you get any solution for this issue?

9

@Rammohan91 :smiley:Thanks Man it actually worked. I unchecked parentid=‘ember*’ and selected some other selector which doesn’t change at all now my program is running fantastically. Once again thanks really appreciate it.

1 Like
10

@xkarrox @Rammohan91:+1::+1::+1:, I have taken @Rammohan91’s suggestion and Selected the most unique selector which doesn’t get changes after refreshing the browser or even if Data get changes and you know what it really worked for me hope it will work for you as well :smiley:

1 Like
11

Hi,

Can you please tell solution for scraping linkedin data.
Every time selector gets changed. I have used uiexplorer also.
but when i selects dynamic selector, it does not extract data. ?

Any solution would be really appreciated.
Thanks

12

@vinsmart06 Please let me know what kind of data you are looking from LinkedIn.

13

Firstly, i am not looking any data.
I wanted to know whats the solution you applied for scraping linked in.
How to scrape email id and phone number from LinkedIn.

15

hey all
i have been working for the same and im trying with linkedin company profile to get how many are
like for e.g.: i have share a SS below
1
after this link i got a list of employee’s
1
after that i clicked on next and the process is running but i couldnt fetch the info and it closes immediately can any one suggest where im going wrong

aim: to get the list of employee’s in a excel sheet

thanks in advance guys

16

again me
sorry for the trouble as when i try to repeat the automation it showing error in one or other activity can u solve this issue and let me know this is my main
Appstekcorp.xaml (11.6 KB)
if any changes and suggestions let me know

eagerly waiting for the rply
regards
newbot in uipath
@Myshudhage