Here is a Challenge.
For the last few months I have been trying to Open a Browser, Scrap two tables using Data Scrap, and write to Excell CSV.
Well… the site link is… Civil Engineering Services & Supplies, Newtownabbey, Property Developers (nifed.co.uk)
I have downloaded all the Business from the Directory, so I have just 4000 url’s for each Business.
I have tried to READ RANGE the sv file with all 4000 url’s for each business.
The For Each Row,
the iterate through the CSV file of each line or Url addressfor each business.
Then DATA SCRAPE the Name of the Company and address and DATA SCRAP the email and website address.
The DATA SCRAP captures two tables,
Then Write to Excel.
The PROBLEM and Challenge.
By DATA SCRAPING and using the “Next Page” activity, only works until the next page selection is no longer available. Say 8 or 9 urls. I have continued on errror, to the next For Each Row url from the READ RANGE File.
The process runs accordingly: However, after a few runs through the sequence. The process only loads the OPEN BROWSER, Closes the Browser, and moves on to the next FOR EACH ROW, missing the data scraping.
This should be relatively simple DATA SCRAPE. But I have been at it for months and cant get it to work?
Any ideas guys.??
Also I have noticed this for many other DIRECTORY sites, where they cut off the NEXT PAGE option, only allowing access to say 30 address It looks like a Websmater solution to prevent scraping.
Good all, and let me know