Data extraction from a web page

Learn
, ,
1

I have a webpage with structured data. Each column has a hyperlink. When I click and go on that page I have two fields to extract. Then I need to come back and open the second link and go on that page and extract same two fields. I need to do this for a hundred links. Can anyone please help me.

2

Hi @Sekhar_M

I already automated a similar process. My approach was to extract the main page with all the hyperlinks. Afterwards I loop through the extract data table and open the hyperlinks one after another.
Now again I extract the hyperlinks from the page opend and loop through the second extract data table.

Cheers,
Timo

3

Can I have that process?

4

Hi Sekhar,
sorry for the late reply. I tried to simplify the process to your needs. You need to adjust it especially the scraping parts and for sure the assign activities.
Hope it will help you or gives you an idea of how to proceed with your process.

Please be aware that I use Studio version 20.19.10.3 and the browser is Chrome.

Cheers,
Timo
Get documents.xaml (26.2 KB)

1 Like
5

Hi Timo,

I am new to UiPath and I have to automate a similar process to what Sekhar has asked for advise in this thread.

I opened the file that you attached but I am don’t know how to update it for my needs.

Could you please suggest me some links with training that I need to look at before automating such a process?

Cheers
Alex

6

Hi @alex17

I highly recommend you the UiPath Academy: UiPath Academy

In particular if you are new and want to get to know Studio you should do the RPA Developer Foundation course.

Cheers
Timo

7

I have a smilar process… but my hyperlinks are not coming into excel as hyperlinks this is my issue The process I am trying to achieve is:

Open browser and go to this link: TSBPA - Welcome to Texas State Board of Public Accountancy
I enter the Firm Name: PLLC
I enter the City Name: i.e., Lakeway (I need to do this for several cities with a lot more than 3)
I add a click action and Submit Search

This gives me a list of lookup results
I need to click on each Firm Name and data scrape the table showing the firm information
(i.e., Firm Licenses, Firm name, Resident Manager, Contact Information, Phone Email Status)

I need to save this information to excel spreadsheet as a table with headers.

I have found tables on HTML pages: Extract and manipulate data but it did not give me the results…I have been able to get as far as extracting the table, but not sure how to make it loop continuously.

8

Hi @uidonna.jones, when you are using data scraping the widget asks you to hover over a data source field. I always select the first row of the table I want to extract. Once you selected a row UiPath prompts a pop up:
image
You need to click no. It will now ask you to click the last row in your table. Please do so. Afterwards you are able to check the URL and rename the column names:
image
Simply follow the next steps. Your output now is a datatable where you can loop through with a for each row and so on.

I hope that helps.

Cheers,
Timo

9

Thanks. I did the following steps, but they do not come in as a url, probably because the website is a php site. Do you know how to convert a path to a hyperlink in excel?