Monitor website and if new data is found, download. If not, ignore

Hi all,

I have a Process that opens a browser, scrapes some data using the data scraping function and when its done, it saves the data gathered into a csv. Would it be possible to tell a robot to monitor the said website everyday and if new data is found, download the whole data again?

For example:
The data im downloading right now has 819 entries (table). If the robot finds out that there are ie 840, download the whole thing again, on its own.

Thanks in advance,

Juan

Hi welcome to the community!
Yes, it is possible to run a scheduled process to go there and check for this, but that is depending on that application provides the counter to be compared, lets say an number asset that you record everytime you run this.

Hi @tsielk,

Welcome to the Uipath Community! :smiley:

Yes of course it’s possible but for that to happen you have to store the data somewhere. It can be an excel, database, csv which you’re doing already the csv. I recommend to save the document with today’s date using the Now function for example in an assign activity you put variable = Now.ToString(dd.MM.yyyy) the output of that will be today’s date with dots “.” now when you scrape which I recommend scrapping everyday because you may have today 300 users and then tomorrow again 300 users but what happens new 10 users where added and 10 where removed? you will still have 300 right? So you will have to match yesterday’s info with today’s you can use the Join Data Tables activity which gives you many options to match the info and decide what you want to do with it.

Combines rows from two tables by using values common to each other, according to a Join rule, which is specified in the JoinType property.

Options

  • JoinType - The type of Join operation you want to use. The following options are available:
    • Inner - Keep all rows from DataTable1 and DataTable2 which meet the Join rule. Any rows that do not meet the rule are removed from the resulting table.
    • Left - Keep all rows from DataTable1 and only the values from DataTable2 which meet the Join rule. Null values are inserted into the column for the rows from DataTable1 that don’t have a match in the DataTable2 rows.
    • Full - Keep all rows from DataTable1 and DataTable2 , regardless of whether the join condition is met. Null values are added into the rows from both tables that don’t have a match.
      More info here:
      https://docs.uipath.com/activities/docs/join-data-tables

Hey! @bcorrea and @rmunro thank you both for your answers and warm welcome.

They both were helpful.

Regards,

Tsielk

1 Like

Can you give some example .xaml file

@rmunro Can you give some example for monitoring the webpages for releases for example in python websites if they released new versions how to automatically received mail