Remove Duplicate Rows but still keep the valid duplicate row


I am trying to figure out how to avoid scraping the duplicate value from the web app and write into excel. I can remove the normal duplicate row but some of the “Duplicate value” are valid from web app. so how do i remember which one is valid duplicate and which is not. Some of the valid duplicate rows can be the exacted same number for all the columns which make me hard to determine.


  1. First use Data scraping activity to scrape the data from web application and will give you output as DataTable and say ‘inputDT’.

  2. And then use below query to remove duplicate rows.

          newDT = inputDT.DefaultView.ToTable(True,"Column 1","Column 2",....."Column N").CopyToDataTable

The above condition will remove all duplicate rows based on columns mentioned here.

we dont use data scraping activity to scrap as our web app is not in html table. we used array object to capture each data field.
that’s the problem with remove duplicate rows. As we may encountered valid duplicate rows i can’t really use that. what i mean by valid duplicate rows are in our data. there could be exact values rows that happen 3 or 4 times. If i use remove duplicate rows it will also removed all the valid duplicate rows.

How to use query in uipath
if I copy this in filterwizard it throws error