Extract data from website

Hi,

This is the website I would like to extract data from:
https://www.dailyfx.com/calendar

Is there way to extract the events that are of high importance (says high in red) and save each data as a variable?

Thanks

Hi @bhe98

You could use the data scraping wizard to extract the data from this page. You can access the data scraping wizard from the top ribbon of UiPath Studio.

This will allow you to scrape the fields from the page as you required. The extraction will be done to a data table variable.
You can then filter the results using Filter Data Table activity to filter the data to high priority records.

You can follow these steps as well

  • Open browser
  • Navigate to web site
  • Data Scraping wizard - this will automatically create another set of activities…
  • Filter Data Table activity to filter the results
1 Like

hello @bhe98

you can extract whole data using Data scraping activity which will return a data table with all the priority data like high medium, and low as well.

loop through it and if the importance is high then store it in a variable.

Regards
Ajay

@bhe98

you can take help of below xaml.

dailyfx.xaml (8.9 KB)

Thanks!

1 Like

I tried data scraping, however, the data extracted were inaccurate. Are there any alternatives?

Hi @bhe98

The initial extraction using data scraping will extract some unwanted data as well. The workflow provided by @Navneetj3 shows it clearly… The initial extraction creates some unwanted data. However, after the filter he has applied, it filters the required dataset for you.

I tweaked @Navneetj3’s same workflow a bit to add the data to a excel file. It adds the full extracted list to a separate sheet, and the filtered list to another sheet which clearly shows the expected result.

Edit: one thing I forgot to say is, the reason for the data scraping to extract some unwanted stuff is because of the structure of the table in the web page. It has some merged cells and some rows are considered as a merged row. I identified that by trying to do scraping for one column. However, filtering it gives you the expected result

dailyfx.xaml (9.1 KB)

1 Like

Thanks for your help.

Another question, what should I do if I want to save one of the data as a variable (i.e. just the name of the event like “CNY manufacturing PMI” )?

@bhe98

In case you want to loop through the data table and access the values row by row to perform certain tasks, you can use a For Each Row activity to traverse through the data table.

Within the loop, you can use a Assign activity to access the values of each row & column

String Variable = row("ColumnName").ToString

This code is used to access the value of the specified column of the current row

Hi,

I tried scraping the data for today, however, it still extracted the data on wednesday. Thus, I compared the selectors between two days.

wednesday: < webctrl id=‘daily-cal3’ tag=‘TABLE’ />
thursday: < webctrl id=‘daily-cal4’ tag=‘TABLE’ />

I was thinking of using dynamic selector. But I also tried:
< webctrl tag=‘TABLE’ class=‘table dfx-calendar-table tab-pane fade active in’ />

  • it will only extract data in the selected pane
  • however, the selector for selected pane seems to be dynamic as well
  • ‘in active’ instead of ‘active in’ sometimes

what do you think would be better for me to do?

Thanks a lot

Hi,

You can use active so that it will work for both active in and in active.

How do you do that ?
Like this ?
< webctrl tag=‘TABLE’ class=‘table dfx-calendar-table tab-pane fade active’ />

Try this
< webctrl tag=‘TABLE’ class=‘table dfx-calendar-table tab-pane fade active’ />

Doesn’t work though.

The italic part doesnt show when entered in

is it possible if i do something like:
< webctrl id=‘daily-cal’ + ‘Date.Today.DayOfWeek.toString’ tag=‘TABLE’ />

Date.Today.DayOfWeek.toString- this would give out the day today like Thursday

is there a way to convert it to integer, in that case thursday= 4

Hey, Can we scrape certain sets of text of data from a live website?