I have a requirement where there is a list of more than 100 different products, we have to scrape the reviews of each of these products from amazon & write to csv. I have designed a workflow where I have fetched the urls of all these products & in a loop, opens browser for each url…scrapes data…writes to csv…close browser. But it has to do this more than 100 times, it’s a time consuming process. Is there an efficient way to do this?
Best and efficient way will be:
Use a loop for all products collection
Open browser only once and other times use Navigate To
Store review in DataTable
End of loop: Write DataTable into CSV
Seems like you have already been successful in doing this and you are looking for a more optimal solution…right? (Correct me if I am wrong)
Then can you upload your project in a zip file and mention the landing page/URL (if you haven’t configured it) ?