How to avoid from duplicates uploading into queue?

[Ayatulla_Middya]

Let’s say, i have an one process built in RE Framework, I am using dispature and performer in single process.
I have uploaded into queue all data(200 items) from an excel.

so i have put this logic about uploading the data into queue, in config is nothing block, in initialization xaml file, so in continue process, it will upload all data, and it will process one by one, right??
if we get any system exception it will come to initialization, it will check config is already initialized, so it will not again upload the data, and it will fetch next queue and it will process, correct??

now, my question is
1> let’s say after succussful 50 items, if system exception occurs, and exceeds the max limits, then it will go to end process, now if i again restart the process, it will upload again all items(if not unique) or those items are processed already(unique based on index, any column based). How to avoid this from duplicates uploading??

[Anil_G (Anil Gorthi)]

@Ayatulla_Middya

1. Enable unique reference in queue and add unique id as reference so that another item with same id is not added
2. Second way check if there are items in queue and do not add if already present
3. if you say there might be new items and some times I mgiht not have unique reference…then you need to use get queue items first with reference filter check if same item is present in new state and if no add it

cheers

[ashokkarale]

@Ayatulla_Middya,

You should have unique reference to avoid this duplication. If you have, then just enable it while creating Queue.

Thanks,
Ashok

[tushar_srivastava]

Hi @Ayatulla_Middya,

• The recommended approach will be to have dispatcher and performer as a separate workflow, if possible.
• Yes, it will upload all the data one at a time. You can also use Bulk add queue items.
• In case of system exception, and when you retry the process, it will be uploaded again.
• To avoid this, you can maintain an Excel workbook with a list of all the transaction items, and a new column “Status”.
• Also, maintain unique reference.
• Add an if condition before uploading and only add those whose status is empty.
• Mark the transaction items as “Uploaded” in the Status column, once they have been added to the queue.

[Ayatulla_Middya]

@ashokkarale thanks for reply but although queue is unique but those are already processed those will be uploaded again right if again restart the process?

[ashokkarale]

@Ayatulla_Middya

No they will not be added again whatever state be it. Bot will throw this exception.

Add Queue Item: Error creating Transaction. Duplicate Reference. Error code: 1016

So make sure you are using Add Queue Item activity inside Try catch like this.

Thanks,
Ashok

[Ayatulla_Middya]

@Anil_G thanks for reply
before uploading i need to fetch queue items with reference
there filter strategy two types one is starts with or equals so, can I provide multiple reference, so that we will get all successful items.
if I can then what possible reference will be, let say, my unique reference is the index then.

[Ayatulla_Middya]

@ashokkarale thanks but again one doubt,
If this is the case, we can’t upload items after successful all items,
means I have 200 items and I got successful, second time I want to run the bot, it will not upload the items into the queue, right??

because in queue all 200 items are successful. now second time I wanna run the bot then??

[ashokkarale]

@Ayatulla_Middya,

Will your queue item reference going to be same again? if yes, then you will have to make the reference unique per day or week or month as per your run frequency.

If it’s going to run everyday then you can make the reference like this

"OriginalReference"+Now.ToString("dd/MM/yyyy")

[Ayatulla_Middya]

@ashokkarale
Or we delete the successful queue items using delete queue items activity then we can add items again, is that a good way of doing??

means it will be a condition, once we will fetch new items from queue before every uploading, we get 0 then only we can delete the successful queue items then we can run the bot at same day again right??

[ashokkarale]

@Ayatulla_Middya,

You can do it but it will loose it’s original status - Failed/Successful.

If you don’t want it at later for root cause analysis or anything then only use this approach.

[Anil_G (Anil Gorthi)]

@Ayatulla_Middya

you need to give each index separately so addition should be done in a loop where first get with reference and check count…if count=0 then add else dont

cheers

[Ayatulla_Middya]

@ashokkarale @Anil_G @tushar_srivastava

“Thank you very much for your assistance. I truly appreciate your help.”