I really appreciate this function, because it offers a clear advantage when reading data compared to the conventional method. However, I miss a function that prevents empty records from being imported as well. Unfortunately, we sometimes receive data that contains blank lines. Then we have problems, because this function then creates empty QueueItems. It would be desirable if you could enter column names here, which must never be empty, otherwise the data record is discarded. Thanks in advance.
I understand your concern.
To avoid importing empty queue items, You need to just clean up your DataTable before you proceed with bulk add queue items.
For cleaning up, you can use
- Filter DataTable
- Query DataTable
Filter DataTable will be more easy to perform as it is activity based, whereas querying needs code to be built.
Hope this is what you are looking for.
Thank you for your answer. That is what we are doing now. We read .csv or .xls* files into a datatable. For example we call that table tblCSV. Here we use this expression with an “assign” activity on the right side:
tblCSV.Rows.Cast(Of DataRow)().Where(Function(row) Not row.ItemArray.All(Function(field) field Is DBNull.Value Or field.Equals(""))).CopyToDataTable()
This filters out emtpy rows and is very fast. After that we also check e.g. if the colum names are containig “.”.
If now everything is fine we use use BulkAddQueueItems.
My suggestion was just maybe to integrate that into this function, because if you have the option to name so-called key columns that must not be empty, that would be very convenient.
Got it @martinN.
Could you also please add
field is nothing and optimise your condition a bit
Hope this helps.
You need a Reference, add one like this example:
dt.columns.Add(“REFERENCE”, System.Type.GetType(“System.String”), “Field1 + Field2”)