Remove duplicates before inserting into database

Help
1

Im using uipath to insert data into a database. I have a field set as a primary key on my table. The data coming from UIpath is duplicated is there a way to tell UIpath to check for duplicates based on the data in the database and the new data. Right now, I get the an error message that says “error on duplicate key”. which is the primary key. Since im not too familiar with UIpath I wold appreciate if someone can ceate a sample video how to do it.
Thank you.

2

@jsmooth

take distinct values.

suppose if you are having an array and if you want to remove duplicates then you can use

array_variable.Distinct assign this to some other array variable. now u will have only unique values in new array.

Regards,
Mahesh

3

Hi @jsmooth,
Refer this post to how to remove duplicate records.

Regards,
Arivu :slight_smile:

4

Thank you for the suggestion. However, I already know that my data I have is not duplicated. My issue is when I add more and more data to the MySQL database there will be duplicates. So I need to compare my data that I just retrieved and compare it to the data that is already placed in the database. My plan is to query the column with the primary key from the database and compare that with the primary key of the data I just retrieved. If anyone knows of a simple way to do that, I would appreciate it :slight_smile:

5

@jsmooth

Try this
If schema of both the datatables are same then you can try this query.
let us take dt1 as sql database and dt2 as ur retrieved data
let dt3 will be the datatable which will be having rows which are not there in dt1 and will be there in dt2

dt3= (From p In dt2.Select()
where ( From q in dt1.Select() where string.Join(“,”,p.ItemArray).equals(string.join(“,”,q.ItemArray))
select q).ToArray.Count<1
select p).ToArray.CopyToDataTable

Else

if you want to compare only on primary key then use this

dt3= (From p In dt2.Select()
where not ( From q in dt1.Select() select Convert.Tostring(q.Item(“Primary Key _column
Name”).contains(Convert.ToString(p.Item(“Primary Key _column
Name”)
Select p).ToArray.CopyToDataTable

Regards
Mahesh