Problem in Data Scraping

i want to pick comments of a product of amazon.i done data scraping for all comments using data scraping with next page. But now some comments are stored in data base.now i want to pick only new comments(which are not present in database.)
how can i do this?
Pls help

Hi @arijit1213

You can use [Data tables comparison] to do that you want.
Please download it from the below link and install it to your local.

Step:
1.Get data from your database by [Exexute Query] => dt_A
*Need to install UiPath.Database.Activities
2.Scraping data from amazon => dt_B
3.Create new datatable variables => dt_C
4.Assign dt_C = dt_A.Clone
5.Using [Data tables comparison] to get new data and put it to dt_C

*Please be careful, data table structure and data type between dt_A and dt_B
must be same.

Get data from the database


Data Scraping

Data tables comparison

I hope this information will be useful for you :blush:

1 Like

thanks for your reply…
Assume in a product of Amazon 5000 comments present in today and i scrap whole comments and stored this 5000 comments in Database.the next day total no of comment will become 5050.
i don’t want to scrap whole 5050 comments again (previous 5000 + 50 new) and then compare it with database or eliminate duplicate data.(because it will take more time)
i want to do scrap only new 50 comments and then store it to data base.
Thank You.