Page Scraping using UiPath - Robots.txt


Can I scrape some details of any webpage using UiPath?

If yes, should robots.txt file of the webpage needs to be checked before scarping the content using UiPath?

Please let me know your thoughts.

1 Like


You can use web scrapping to get the details from webpage you can store output as string and use write text file and you can take output in .txt