Hi, we have an in house web application. The workflow we need to do is go to a page on the application > scrap the data that appears inside many different input fields > we then need to output that data to a .xlsx file. We started with a dynamic selector of the fields to scrape but the problem now is, once the initial lot of data is done scraping we then need to go to the next page and scrape more data if it’s there to be scraped. This part is troubling because there could be an unknown amount of next pages to go to because it’s based on how much data is in the database on that day. How can we select and get the data from input fields on another page if we don’t have the selectors?
Note: each next page we go to looks the exact same but I have no doubt has a different selector.
I am new to this and this is very troubling. Please help
Is the Field name which you are scrapping same for all pages ?
Yes. I am not in the office yet but to give you an idea. The input fields have only one label field name on the first row. Eg:
Name:
[Field]
[Field]
[Field]
[Field]
I wish each input field had a field name above it but they dont. Only the first row
I was going to code up a webpage that looks similar and make public to show people but I thought I’ll ask first
If the data you want to scrape is structured then use “Data scraping” activity it has an option to choose next page.
Thank you but unfortunately it’s not structured at all and the data scraping wizard won’t scrape the data from the input fields because they aren’t a table of fields.
Any ideas how i can do this?
Assuming you have to click somewhere to go to next page, you can use element exist on that element and if it exist click it and scrap the data, and continue to do it in a while loop until the element exist. At last page there should be a change in that element selector.