Webscraping the same website but through a loop

Hello! I am currently a rookie and learning how to use UiPath.

I am working on a project where I am given a list on excel, have to iterate through each record and web scrape a website to search the results for each record. The problem is, the ‘for each’ loop worked in opening each of the webpage, but UiPath seems to have scraped the webpage for the first record. Is there any way I can go about to fixing the problem? Thank you!

Please elaborate more…
If the data has been scraped on the first loop then that should be okay for the rest.
Where/how do you stor ethe scraped data?

Hi @ktanli

After first record of you are scrapping something and storing it an any variable that can be append to a temp variable and you can move to next record.

If you assign scrapping data Into a temp variable then it will append each time of scrapping data.

Thank
Latika

Sorry, I felt like I needed to elaborate more.

  1. I have a list of zip codes that I need to iterate through and type into a search bar of a website that spits out schools in the area.

  2. The thing is, when I did a for-each loop, the ‘opening of webpages’ by zip codes worked out beautifully

  3. When I started scraping the data, it was based on the first zip code on the list.

  4. Somehow, the other zip codes had the same results as the first ( it was not supposed to happen this way)

  5. I made the process in such a way that:
    a) UiPath gets a zip code from the excel file
    b) enters it in the search bar
    c) scraped the school name in the page
    d) The results are then generated in a new sheet of that same excel file.

Hi Latika,

So basically if I have 10 records, I have to have 10 temp variables? What if someone asks for like more than that? Please let me know! Thanks!!!

@ktanli

You have to check the selectors, You have to make the selectors dynamic to get different values

Can you share your .xaml file and project.json files so that we can check

Thanks

No Just take 1 temp variable and What ever the scrapping the variable can add to temp variable, you will get all scrapping values into one variable.

For ex: TempVariable = TempVariable+ScrapedVariable

Thanks
Latika

Hi! Sure thing! This was the latest version of the process I was working on. The zip codes are in the excel file (Sheet 1). The sheets that are labelled as zip codes are basically what happened when I ran Trial 4.

school.xlsx (14.2 KB)

Trial_4.xaml (12.1 KB)

Hi @ktanli

Its working fine now. It is scrapping the data from every zip code and then storing the scrapped data for that particular zip code by creating new sheet by the zip code name.

Below is the updated workflow.
Trial_4.xaml (15.8 KB)
school.xlsx (15.3 KB)

Output :-

image

Mark as solution and like it :slight_smile:

Happy Automation :raised_hands:

Best Regards
Er Pratik Wavhal :robot::man_technologist:t4: :computer:

Hi Pratik,

So what exactly happened? From what I gathered, I changed the selector ( in the title part) to be all wildcards, I did that to make it dynamic ( was that right?)

Was the problem that for the loop I did not add the close tab activity?

Hi @ktanli

The problem i encounter that the data which was scrapped for 1st Zip code was not getting cleared and when the loop start executing same thing for 2nd zip code datascrapping was not able to store the new scrapped data.

So before going to 2nd zip code i have clearing the datatable after writing the scrapped data to excel for 1st zip code.

Hope you understand this. If this helps you

Mark as solution and like it :slight_smile:

Happy Automation :raised_hands:

Best Regards
Er Pratik Wavhal :robot::man_technologist:t4: :computer:

Ohhh… So I basically have to add another activity to clear the data table after publishing the results on excel in the for loop?

Hi @ktanli

Yes. Bcz i find the issue was der only.

For that Clear data table activity is der.

Mark as solution and like it :slight_smile:

Happy Automation :raised_hands:

Best Regards
Er Pratik Wavhal :robot::man_technologist:t4: :computer:

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.