Unable to scrape more than 10 tweets on twitter using the table extraction wizard

Help Studio
, ,
1

Hi, I am a beginner to UiPath and have been working on a bot that scrapes tweets on a particular hashtag but using the table extraction wizard im unable to scrape more than 10 tweets despite setting no limits on the extraction. However, I noticed that upon setting the browser zoom to 25 percent im able to scrape around 20 tweets which is a change from the usual 10. Any ideas on how to fix this?

2

while using data scraping select multiple pages
by clicking next symbol

3

image
image1920×1080 149 KB

enable extract data from multiple pages then indicate next button on the page

Hope it helps,
keep learning
Happy Automation

4

hello, thanks for getting back. Twitter doesnt have an interface where i can select a next button, it keeps scrolling and loading new tweets infinitely. What shall i do in this case?

5

try by changing in properties input mode to simulate or window messages

give time out

Hope it helps

6

ive tried that and set a timeout of 5 seconds and unfortunately it didnt work

7

@Harith_Sai_Saraf

Try scrolling till bottom and then extra the tweets

There shpuld be some break to scroll ypu cant expect whole intimite scroll to be extracted

Cheers

1 Like
8

hello anil, just set a limit of 60 tweets but it still scraped only 7

9

@Harith_Sai_Saraf

Its not about limit…but did yous croll and then extract?

Cheers

10

Added a mouse scroll activity to automate the scrolling process and it didnt work, also tried scrolling manually and extracting as u suggested but it still didnt fetch over 10 tweets

11

@Harith_Sai_Saraf

When you say as it scroll it loads, is it like the previous ones are also not cisible…can you inspect the element ot view pge source and check if there are only 10 items always…

If so then we need a loop of extraction and scroll

Cheers

12

Before the extract activity, Send Hotkeys Page Down as many times as you need to get as far down as you want.