First post and if this has been answered to some great detail I’m so sorry.
I’m trying to find a way to mine the information of Child pages, on let’s say, Amazon.
As opposed to just grabbing all the summary info, I want to open each link in the search, and mine the details page for the information. This seems very doable but I’m stumbling.
My current Plan of Attack was to set up a flowchart that opens the browser, takes your inputs for the search, then scrubs the results for the URL of each result and dumps that into a dataTable? CSV? (I’d like to keep it in the tool and not generate mess). Then read in the first URL, type into the URL Path, scrape the details, write to CSV, do while there are still URLS. At least that’s my thinking…
Any help would be greatly appreciated; what a tool this is!!!