**CHALLENGE** DATA SCRAPING

Hi Team,

Here is a Challenge.
For the last few months I have been trying to Open a Browser, Scrap two tables using Data Scrap, and write to Excell CSV.

Sounds Simple?

Well… the site link is… Civil Engineering Services & Supplies, Newtownabbey, Property Developers (nifed.co.uk)

I have downloaded all the Business from the Directory, so I have just 4000 url’s for each Business.

I have tried to READ RANGE the sv file with all 4000 url’s for each business.
The For Each Row,
OPEN BROWSER
the iterate through the CSV file of each line or Url addressfor each business.
Then DATA SCRAPE the Name of the Company and address and DATA SCRAP the email and website address.

The DATA SCRAP captures two tables,

Then Write to Excel.

The PROBLEM and Challenge.

By DATA SCRAPING and using the “Next Page” activity, only works until the next page selection is no longer available. Say 8 or 9 urls. I have continued on errror, to the next For Each Row url from the READ RANGE File.

The process runs accordingly: However, after a few runs through the sequence. The process only loads the OPEN BROWSER, Closes the Browser, and moves on to the next FOR EACH ROW, missing the data scraping.

This should be relatively simple DATA SCRAPE. But I have been at it for months and cant get it to work?

Any ideas guys.??

Also I have noticed this for many other DIRECTORY sites, where they cut off the NEXT PAGE option, only allowing access to say 30 address It looks like a Websmater solution to prevent scraping.

Good all, and let me know

GAz

Hello @DiBNI_Digital_Insurance_B!

It seems that you have trouble getting an answer to your question in the first 24 hours.
Let us give you a few hints and helpful links.

First, make sure you browsed through our Forum FAQ Beginner’s Guide. It will teach you what should be included in your topic.

You can check out some of our resources directly, see below:

  1. Always search first. It is the best way to quickly find your answer. Check out the image icon for that.
    Clicking the options button will let you set more specific topic search filters, i.e. only the ones with a solution.

  2. Topic that contains most common solutions with example project files can be found here.

  3. Read our official documentation where you can find a lot of information and instructions about each of our products:

  4. Watch the videos on our official YouTube channel for more visual tutorials.

  5. Meet us and our users on our Community Slack and ask your question there.

Hopefully this will let you easily find the solution/information you need. Once you have it, we would be happy if you could share your findings here and mark it as a solution. This will help other users find it in the future.

Thank you for helping us build our UiPath Community!

Cheers from your friendly
Forum_Staff