I have a large number of websites in an excel all having different structures and I want to extract (like name and date) certain data from all of the websites , but can’t scrap each field in every website separately as there are too many fields and too many websites as well.
I am able to scrap whole data from a page and write it in a notepad but then also can’t fetch the values required from that data.
I would like to know if there is a way to do it dynamically?
Welcome to our UiPath Forum!
I am afraid dynamically recognizing values from a wide range of available data is something that is not easily done. UiPath excels at automating specific, well-defined processes.
One idea, one that you already recognized, would be to try to use Screen Scraping on the entire page (instead of saving it). You would then end up with a big block of random text, but maybe with some string post processing (regex?) it would be doable.
Thanks for the reply .
Already done that but that is not of much help.
Can you please suggest a way , if any , how we can iterate through the menu items in all these dynamic websites where we only have the URL?
My first instinct is that it is very difficult to do. Or rather, that UiPath is not the right tool for the job, given it is a process that is not well defined from the start.
Maybe somebody else will be able to suggest some solutions based on their experience. I suppose reading the text is one way. If you could somehow combine it with first using some other software to download the pages (including content from links on a specific page), and only then run the Screen Scrapping, then maybe it would be of help.