Queue / Transaction Management Improvements

i_completed
orchestrator
r_2018_2

#1

2018.1 version implemented query queue items and delete queue items. That is great but are there any plans to upgrade queue management to be able to

  • Query transaction items: Get Transaction Item that meets a specific filter. If no such item, we would get empty queue item (or an error) else we get an item that matches the filters

  • Update and Put back an Item to Queue: Now if I get a transaction item, I must set its result to either Success or Failed (AppEx/BizEx) but it is needed to put back an item to the queue with an updated status for later processing. And next time an other job / robot can process it. I have heard about setprogress but this works only if an item is at me and I can not put it back to the queue anymore, so the current job must finish it to success/Failed.

Anyway UiPath is very good :slight_smile:


#2
  • for querying the items - GetQueueItems activity
  • to put a transaction item back there is PostponeTransactionItem

I thing what’s missing is a GetTransactionItemByReference.


#3

Yes, GetTransactionItemByReference would be enough to fight for a desired item. That would be great if if this field would be updatable and be able to put back to the queue for later processing.

PostponeTransactionItem puts back and item to the queue to a certain time and not to the end of the queue. If GetTransactionItem returns nothing, the robot can not decide whether the queue is empty (so all items are processed) or there are items that are postponed and the currently issued GetTransactionItem is too early to get those items.

My problem is that current queue management system in UiPath considers transactions as “atomic”. If a robot gets a transaction item, it must process it 100% and the result can be either success or failed. However in most of the cases every item in the queue must pass several steps (do something, save in erp, do another thing, save in erp, do third thing, save in erp, send an email an then done, and this is the point where it can be succeed.) After saving on the first screen the item can not go back anymore to the first screen.

What to do now if the processing fails at step 2 for watever reason (gui timeout for example). The queueitem is set to failed and then it gets back to the queue for retry. Next time when it becomes a TransactionItem again (by another robot maybe by another time) the robot will not know that this item already passed the first screen and needs to go to the second.

In my oppinion this could be handled by an updateable field (e.g. milestone) in queue management:
Get Transaction Item, check its milestone, based on do the processing at the right step.

  • When that step succeed
    If this was the last step, set transaction to succeed
    If not, then set the queueitem’s milestone to the next and
    …Put back to the queue or
    …Continue its processing on the next step.

  • If failed by AppEx, leave the milestone as is. The next time it will point still to the missing/failed step.

In some other cases transactions can go to the next step only if some other items already passed the n.th step.

These cases can now be handled by

  • Keeping an excel file and there we update and maintain statuses. In this case the job can be executed only by 1 robot at a time since it can not be read/open simultaneously.
  • Using multiple queues for each milestone.

#4

This is correct. That being a queue mechanism if we put back the same item then it will be the first not the last. In order to put it back at the end of the queue we’ll need to create another containing the same specific data. This is what retry does in case of app exception. We’ll think about put an item back at the end of the queue.

I think this is mitigated too.

  • On the transaction item you have a field named progress which you could customize in any way (let’s sey you add a string during processing STEP1-OK, STEP2-FAIL).
  • when application exception occurs you will have a fail with application exception
  • however at the queue level you have a retry number, therefore the transaction will be retried by another robot
  • the second robot will read the progress field and it will jump directly at step 2

#5

Badita, thanks for your reply.

Yes, set progress can be a workaround in this case, since transactions remain in “InProgress” status even if they are set to AppEx - somehow I have not focused on it yet. However this needs the queue to be set to a high retry level.

Anyway, thanks for this hint.


#6

A short note…don’t mix the “InProgress” queue item status (new, inprogress, failed, retried etc) with the field “Progress” which is a string that can be set as needed by the developer. I feel we should document this better :slight_smile:


#7

Yes, its clear. Why I was writing that is because it is written your docs, that SetProgress (string) can be used only if the item is in “InProgress” status. In my mind (I do not know why) inprogress status menant “the item is between GetTransactionItem and SetStatus”. But now I realized that Set Status to AppEx leaves the item in Inprogress status until the retrycounter reached.
So SetProgress still can be read/write until it gets finished.


#8

As far as I know it sets it to Retried if retry count isn’t reached and Failed if it is.
If Retried, it creates another item in the Queue, with Status = New and Parent = previous transaction ID.

Separately, if you have items that are multi-step, consider using multiple queues. Transactions being atomic is right now by design, I’d be worried if that gets circumvented.


#9

Thanks for your info. I can not test it right now. But according to Badita AppEx -> creates new queue item the progress field still queriable (also copied). If this is not the case then I would rather recommend to create a milestone field, since multi-step transactions need so many queues. So it is an other “workaround”, not a solution.


#10

Yes, the progress field gets copied, I was referring just to the part that the item is left in InProgress state until retries are exhausted.

It all boils down to what is a transaction in that process. In ways, there are actually multiple of them and treating it as one is limiting. But that depends on how the process is designed, so I’ll end here to not derail further :slight_smile:


#11

Thanks for your reply.


#12

Btw, thanks to @andrzej.kniola for Postpone and Progress fields. He pushed for them. :slight_smile:


#13

GetTransactionItemByReference will be delivered in 18.2.

Please note the difference between GetTransaction…(changes the status of the item) and GetQueue (does not change the status)


#14

Great news. :grinning: Menawhile we were able to dynamically create queueitems by serailization of dictionary(string, string). With this we were able to generalize the QueueItem creation (and recreation with new status, that can be added to the reference field as well). So multistep transactions can be handled.
It would be great if (future) GetTransactionItemByReference would accept “like” and not only exact match. With that we can spare a GetQueueItems (like) activity to obtain the reference we are fighting for.


#15

Don’t think so. GetTIByRef is designed to return one item and set it to InProgress. Reference unicity should be also enforced in order to make it work.


#16

Sure to return only 1 item (or nothing).
But if I would like to process items where reference starts with “13” (any of them) then I can not do that in this case. The only way is to use GetQueueItems with “Reference startswith 13” then I pick up the first item from the returned collection and pass its (full) reference to the GetTransactionItemByReference activity.