Code taking long time to execute

[Ritika_Singh]

Hi All,

I have a code which I use to remove already processed records from the new records. But with time as the processed record count is increasing it is becoming slow in processing. I am using the below code for your reference ,
completeDataIvosDT.AsEnumerable().Where(Function(row) Not processedDataDT.AsEnumerable().
Select(Function(r) r.Field(Of String)(“Claim Number”)).Any(Function(x) x = row.Field(Of String)(“Claim Number”))).
CopyToDataTable()

Need your suggestions how to make it fast because right now 40k data is there it takes 25 mins to run this. Please help.

@Yoichi any suggestions on how to split these and do something.

[supermanPunch (Arpan)]

@Ritika_Singh

completeDataIvosDT.AsEnumerable().Where(Function(row) Not processedDataDT.AsEnumerable().
Select(Function(r) r.Field(Of String)(“Claim Number”)).Any(Function(x) x = row.Field(Of String)(“Claim Number”))).
CopyToDataTable()

Hi @Ritika_Singh ,

Could you maybe try by creating the prepared data for the comparison ? As it seems from the code you are trying to compare only one column from the other datatable.

We could get the Data prepared for that and stored it an array or a list, then perform the comparison on the list prepared. This way we should be able to reduce some time in Always iterating through the datatable and grabbing the values again.

claimNumbers = processedDataDT.AsEnumerable().Select(Function(r) r("Claim Number").ToString.Trim).ToArray

Here, claimNumbers is a variable of type Array of String

Next, we can use this array in the same way in the Linq that you have mentioned.

completeDataIvosDT.AsEnumerable().Where(Function(row) Not claimNumbers.Any(Function(x) x.Equals(row("Claim Number").ToString.Trim))).CopyToDataTable()

Let us know if this reduces the time for execution.

Also Check the below post for handling errors on Direct conversion to Datatable.

[FirstAid] Handling of The source contains no DataRows exception

This FirstAid Tutorial will describe how a the source contains no DataRows EXCEPTION can be handled. Introduction Let’s have a look at the following filter data table scenario: Name CCode Tom US Charlotte FR Seema IN Jean FR Assignment: Filter all rows on a particular Country Code like FR, UK, ES, UK… Ensure that all CCode data column values are trimmed Ensure that the Filter check is case insensitive we can implement it e.g. with the help of a LINQ statement: dtData.As…

[ppr (Peter Preuss)]

Have a look here for optimization analysis steps

LINQ query running for long hours

Let us introduce the next changes [grafik] we are creating lists with the concatenated strings from dtold, dtnew and creating a dictionary of only the items common in both lists dtNewSet.asEnumerable.Select(Function (x) String.Join(“#”, arrColSet.Select(Function (k) x(k).toString.ToLower.Trim))).Distinct().toList dtOldSet.asEnumerable.Select(Function (x) String.Join(“#”, arrColSet.Select(Function (k) x(k).toString.ToLower.Trim))).Distinct().toList ListDT1.Intersect(ListDT2).ToDictionary(Fun…