DataScrapping on an infinite scrolling page(twitter)

Hey and hello everyone. At the moment I’m trying to create a robot that will take all my followers from a twitter profile and insert them into a csv (preferably the link to the respective profile). My problem is that I cannot add all the values (links).I tried to use the scroll activity together with Table Extraction, in a Do-While… but without success, it doesn’t extract everything. Any idea why? Thank you very much

Write a Do-While that scrolls all the way to the bottom first, then use Table Extraction.

I don’t really understand exactly, would it be too much if I ask for a picture? :slight_smile:

I don’t have any way to give you a picture. Figure out an element that appears when you get to the end. Create a Do While with a scroll and then Check App State inside it. The Do While should continue while the Check App State returns false for the existence of the end element, scrolling more each time.

Then you’ll have all the data on the screen and can extract it.