Scrape URL link of latest article from news site

Help Robot
1

image
image1846×582 68.3 KB

Hi all! I’m trying to scrape only articles dated 9 September (the most recent date) and NOT 8 September 2021.

Is it possible to specify which articles I would like to scrape the URL links of?

Thanks!

2

Hi

Welcome to uipath forum

Better you can scrape all the data with DATA SCRAPPING and get the output as a datatable

While datat scrapping make sure you are including URL as a column in that table

Now in that datatable you can filter like this with a simple expression in a Assign activity

dt = dt.Asenumerable().Where(Function(a) a.Field(of String)(“yourcolumnname”).ToString.Contains(“Now.ToString(“MMMM dd”)).CopyToDatatable()

Which will give the rows with file dated today

Cheers @hailey7

1 Like
3

Hi Palaniyappan,

Thanks for your reply! From your suggestion, I see that Data Scraping is used.

However, when i use Data Scraping, I see that I can only extract two types of values – Text, URL. I’m unable to extract one more value – Article Date. Without the Article Date value, I would not be able to do the Assign activity code you suggested.

May I ask if there’s a workaround to this - and how I can extract the date values in a third column? Or does my Data Scraping activity limit me to only 2 columns?

(Image below shows that only 2 columns can be extracted – title, URL link)

image

Many thanks!

Hailey

1 Like
4

Not really you can select n number of columns
And to add that data as column while data scrapping include one more column and click next and try to select that data section as a element
It will get extracted

This would guide you to get such correlated columns

Cheers @hailey7

1 Like
5

Hi Palaniyappan,

Thanks very much for the link and for your swift response. This helped me to extract more than 2 columns. I’ve since extracted the “date”. However, from the site, the “date” is part of the “description”.

Below is a screenshot of what i’ve extracted as the “Description” column.

Sorry to deviate from the original post, but would you be able to advise if UiPath can extract just the date “September 9, 2021” in the excel sheet itself?

Please find below a screenshot of the “Description” column in my excel sheet. I would like to extract only the date “September 9, 2021” into a new Excel column called “Article Date” so that I can subsequently filter the Excel column “Article Date” in descending order.

image
image1646×675 41.2 KB

Many thanks once again,
Hailey

1 Like
6

Fine

So once after getting this datatable use a FOR EACH ROW activity where pass the above datatable as input and inside the loop use a assign act like this

CurrentRow(“Description”) = Split(Split(CurrentRow(“Description”).ToString,”/“)(0).ToString),”on”)(1).ToString.Trim

This will give only the date and it will assign to that column itself

So the loop will make the above expression to perform on all the rows under that column

Cheers @hailey7

7

Is this working @hailey7

8

hi! Sorry, no it’s not…

9

May I know the issue you are facing
@hailey7