How to data scraping comments which are not present in database

Learn Academy Feedback
, , ,
1

hi…
i want to pick comments of a product of amazon.i done data scraping for all comments using data scraping with next page. But now some comments are stored in data base.now i want to pick only new comments(which are not present in database.) and insert the new comments in db.
how can i do this?

2

@lakshman please check this.

3

@arijit1213 Firstly, I would say, you would have to get all the comments from the Database as a Datatable. Then with the Data Scraped datatable we could get all records which are not matching the ones in the database, then insert those records into the Database as a Datatable.

  1. Using Execute Query method to get Comments from the Database as a Datatable.
  2. Using a Linq query (mostly) to Perform a operation between two datatables to get the Non Matching Records from the Scraped Datatable.
  3. Convert the Non Matching records to a Datatable .
  4. Insert the Non Matching records to the Database using Insert Activity (Probably).

But we would have to know the Schema of the Table in the Database along with it’s types and the Schema or the Columns in the Scraped Datatable, to help you further.

1 Like
4

hi @supermanPunch
can you send me the workflow?