How to scrap the social media(like twitter,fb and etc ) news feed data using data scrapping?

Help
,
1

Hello Every One,
I want the scrap the data from some twitter page. every time i’m getting only first 20 or 40 records.if i do page down or page end i’m getting more records but its scrapping previous and new records.can any one help me to get the total data from this type of silent scrolling pages.

Thanks in Advance.:sunglasses:

1 Like
2

click this : About Data Scraping

to be honest, i don’t understand what issue that you encountered.

but i will give you some important basic info of data scraping tool:

  1. set the Maximum number of results to zero (0) so your scraping will not be limited. Default is hundred (100) means your scraping is limited to 100 rows only. After that u click finish, if still want to scraping more column data then click ‘Extract Correlated Data’

    1
    1819×831 15.6 KB

  2. After you click finish, this window will appeared and asking if you want to indentify the element that navigates to the next pages(?)

  • in your case, if the twitter page have a ‘next’ button, then click yes and indicate the ‘next’ button. So the next page data will be scraped too.
  • But if the twitter page is don’t have ‘next’ button (means only scrollable) then click no, so the scraping will only scraping on that page only.
    2

Hope this can help you :slight_smile:

3

this actually doesn’t work. I just tried it. It just gives the results which are preloaded on the screen and doesn’t navigate further to load new tweets.

4

Hello wija Thanks for the replay.
But Twitter page doesn’t ve next page or navigate button. it works on silent scrolling Like page down only

5

You might want to investigate using the Twitter API connectors available through the UiPath packages:

6

Hello ronanpeter,
I don’t no how to pass the input authentication Details
Get Tweet: Retrieve the Tweets matching the SearchKey into a DataTable.
can you explain in Briefly.

7

can anyone help here with how the input authentication works?

also what would be the tweet id?

8

https://go.uipath.com/component/connector-for-twitter

There is documentation on the connector here.

There is also a chance to contact the developer himself for info.

9

Thank you@ronanpeter ,
I 'll try it once