Looping through links on a website

Help Studio
, ,
#1

i need to click on each of the links below and scrape the information that exists in each of these links. i’m good with automating the info once i click on the links but my question is what would be the best way to loop through all of the links on the first page? here is the process:

  1. click on the first link, scrape the info needed
  2. go back to the page (screenshot below) and click the next link
  3. perform this same process until completed
    Can this be done straight from the website?
    image
    image1209×721 43.4 KB

Thanks,
Deaser59

#2

Hi @deaser59 !
It might be done easier IF you are able to use Data Scraping on this page:

image
image782×138 11.2 KB

If yes, when you click on data scraping, first click on your first element, then click on your second element, and check if the bot gets all the names, then check the option to get the URL:
image

Then what you will have to do would be

  1. Scrape the data & get the URL in a datatable
  2. Use foreach row in datatable and use navigate to activity to go to all the URLs listes to get the wanted data
#3

i’m not getting other URLs when scraping. Any suggestions?

image
image684×682 22.6 KB

#4

@deaser59 ok there are other tricks:
open your two first links (like PIM ESB API Banking and for example) and check the URL to see if there are common elements, something like:
https://www.*/PIM-ESB-API-Banking and https://www.*/PIM-Security-Engineering
The goal is to find common elements so you create your own dynamic URLs.
Then you would have to:

  1. Scrape the titles
    2 use a foreach row and assign with a dynamic URL that contains titles
#5

could you possibly give me a small example of how the for each should look like with using the dymanic url?

#6

Sure ! Would you mind sending me your url so I can make it close to reality ?