Hi, nooby here,
I am using internal system where new data sets are posted constantly.
What I need to do is to monitor for new posts and act on them. Not sure of the way, so I came up with this logic:
Scraping repeatedly (every 5 minutes) a list of records from the web and adding latest 5 to csv by using “append to csv” activity. Once every 5 records are added to csv, I need to check if those are unique and if so I will use them for further work, if not will ignore them.
So the question is how do I determine if those records are unique?
Also the question if this approach is okay, or is there a better way to monitor and act of new posts?