Condition to check previous datatable with currentdatatable while datascrapping

Help Studio
, ,
1

Hi ,

I need condition to verify the current data table with previous data table while scrapping from web application.

While scrapping data across pages from first page we get data and when I go to next page in loop and scrape the data it should not be same .

if it is not same then I can follow with other steps, if the data is same I should retry data scrapping

2

Hi,
If you don’t want the same data inside datatable after scraping can you just check for duplicates in dt after scraping and when you find them just run scraping one/x more time?

3

Hi @karthik_kulkarni1

Scrape and Assigned it into currentDataTable and after that assigned it into previousDataTable

  • Use IF Activity with below Condition

Not currentDataTable.Rows.Cast(Of DataRow)().SequenceEqual(previousDataTable.Rows.Cast(Of DataRow)())

THEN

  • Data has changed, proceed with further steps
  • Scrap data from the current page

Else

  • Data is the same, retry data scraping
  • Handle retry logic

Hope it will helps you :slight_smile:
Cheers!!

4

I need while scrapping , I understand that we can remove duplicates but my scenario I have issue with web app

5

Unfortunately it’s not possible to do that during ‘Scrapping’