Orchestrator Queue downfalls or when to not use it

Help
,
1

Just wondering if there are downfalls / times where you would not use the orchestrator queue. As we have developed up until now using internally developed solutions, namely for loops with status updates to a file for retry. I am interested in the of the pros and cons of the queue with both using the RE Framework or other smaller simpler projects.

Any insight on this would be greatly appreciated. Thanks!

2

There really aren’t any downsides in using a queue, other than it may take slightly longer developing it to use a queue for a linear process.

3

This is using version 2018 so I’m not sure how much has been updated.
and just my opinion…

Pros:

  • better analytics of items’ performance
  • unlocks multi-threading (running process concurrently on multiple robots)
  • more organized info for each item that was performed

Cons:

  • the feature is unpolished and updates could potentially impact existing processes
  • very hands on, unless you make some workarounds. Like, postponing previously failed items if desired, clearing stuck “in progress” items only for first robot, managing deadlines, and failing items that can’t be processed so it doesn’t break the process
  • high learning curve to understand and develop in
  • how do you determine that the last item is being performed, so you can send a confirmation notification to the user?
  • how do you update spreadsheets when all you have is a queue item? Are you content in updating a file one item at a time, and losing any formatting options potentially? Best “known” strategy is to use Workbook Write Range and Append Range inside a Retry Scope (so multiple robots can fight over the file successfully), but sacrifice formatting.
  • dispatching the items to the Queue and managing unique references poses complexity.
  • setting the unique reference requires some thought, because some processes may want you to have a new item based on when a file was modified and some may want the item more unique than that. And, if you have multiple users using the same process, you may want to add the folder name and/or user who last saved a file in the reference.

The thing is though, all those Cons can be solved. So, I believe the Pros outweigh them, and I recommend using the Queue exclusively.

Even if you only need to add each day as the queue item, you can see which days failed and see the information that was outputted for those items. So, it’s beneficial for even simple processes.

Multi-threading, aka running the same process multiple times at the same time, makes it more challenging to get right, however.

You will eventually want a shell (framework) that has all the components you need to successfully manage the Queue with all these cons, and to make it very “hands-off”. Things like clearing “in progress” items when the number of running robots for the process = 1 is a must! Having the option to reset Failed items is also good for processes that run daily and may want to retry the item the next day. Creating a dispatcher that can see the data type in an array and add the items in based on the data type, makes this very easy to accomplish across all various processes. EDIT: Also, having the option to Start the job on additional robots - I suggest using an Argument for Number of robots and Environment, then it can easily be changed for the provisioned process. The Start Job would need to only trigger 1 Robot I think and check the number of robots running the process beforehand; this would need to be after the Get Transaction Item to avoid any loop mishaps.

Eventually In the end, the only things which you would need to change for a new project would be any config files to set all the settings, initializing the dataset to send to the Dispatcher (inside a ForEach if it requires multiple files), and finally the process workflow that performs the steps for a single transaction item (along with any reusable workflows/libraries that are needed).

Regards.

2 Likes
4

Thank you for the awesome reply! Do you believe that the RE Framework should be used exclusively as well? Or just for larger projects?

5

If you are developing for business, I believe you should use a Framework for all projects. The RE Framework is a solution to get started on. You want your project design to be standardized, but in a way that also allows for optimizations to happen. Like if there are things you think in the Framework should be upgraded, upgrade it.

Ideally, you want the Framework to be like a shell, where all you need to do is provide it the process. This can allow simple projects to be completed and perform reliably in under a day, once the Framework is in place.

So, yeah, I think it should be used for all projects.

6

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.