Automatically run bots on new queue item

Hi,

Since 2018.4 adds support for WebHooks, this can now be implemented outside of Orchestrator.

I have created a sample node.js application that does just that. It will use a settings file that links a queue to a process.

It is still work in progress, so feedback is welcome’d:

6 Likes

Nice stuff Bogdan :slight_smile:

2 Likes

Hi,

Current queuing mechanism in UiPath needs processes be constantly checking (pulling) a queue in order to know if there is a new item to process. This seems less than optimal as if there are no items in the queue, the process is just waiting for a new queue item to appear, instead of servicing other processes or queues.

My proposal is to have a new kind of queue, a “push queue”. In this kind of queues, the queue would be linked to an environment (a group of robots) and a process, and as soon as a new item enters the queue, the orchestrator checks which robot is available in the environment and effectively “pushes” the item to the robot, starting the designated process in the robot to process the item. This would be done while there are items in the queue. The orchestrator would follow a round-robin sequence, so all the robots would have the same average load, and all the push queues would have the same chance to push work items to an available robot.

This would optimize robot availability, as different push queues (different processes) could work with the same environment (same set of robots). This would also allow executing a set of processes in a sequential order, just having process A adding a queue item to the push queue associated with process B and so on.

Regards,
Jose.

7 Likes

Overall, I believe this is the best long term solution - “Push Queues”, if it can be implemented by UiPath. The only other good workaround listed above would be that of “Coordinator Robot” that is continually polling queues via Orchestrator API.
I think most of the other workaround solutions, including the new WebHooks, are not efficient methods of achieving the initial goal of automatically running Bots when a new Queue Item is added. Problem, for example, with WebHooks is that you can only setup 1 single Hooks for adding Queue Items, therefore your Web API POST Method is triggered for All Queues in Orchestrator. You cannot limit or designate Adding of Queue Items to a given, single Queue. If you are adding hundreds or thousands of Items across your Queues, then you are going to swamp your Web Service. Not only that, but then if you are Triggering Start Job, then mentioned above, First Job runs, then 2nd goes to Pending and anything after that will not initiate a Job - problem!
Overall, just not a good design to try and trigger Jobs per Queue Item, better off running a Job that can loop and pull/process all Queue Items and then stop. Maybe use the WebHook to initial the Job, but then as others have stated, on subsequent triggers, check the Job Status first, if it is already running then ignore the POST API Call, so you are not processing each POST at a queue item level, but just processing the Queue as a whole. Initiate the Job and let it pull and process all queue items while it runs. Once it completes, then the next new item placed on the Queue will trigger another run of the Job.

1 Like

For this scenario, as a workaround we finally implemented this “Coordination Robot” (we called it “Master Scheduler”), that enables or disables certain configured schedules depending on the queues having items or not, just using the Orchestrator API. We use schedules because that way we can configure which process and how many robots we need consuming items from the queue (the same concept as robots+process from my original post).

It’s working quite well for us. Obviously, it would be much better if this functionality came integrated into Orchestrator, as suggested.

2 Likes

Will be implemented soon with these high level specs:

A queue will have associated no more than one “ consumer” process

At the Consumer Process level we may enable “activation”.

Once the activation is enabled we set the following properties:

**trigger on new queue item

  • **max number of simultaneous jobs - default 1
  • **min number of queue items to trigger a job (don’t count deferred ones) - default 1
  • **ramp-up - default 1 - start 1 bot per x queue items
  • **Non Working Days Calendar (optional)
12 Likes

That will be awesome - thanks!

1 Like

That’s great to hear, I look forward to it!

1 Like

+1 from me.

1 Like

Great New!

~Diego Turati

1 Like

@badita Great idea. You can also integrate Process/Queue priority per Bot into this solution. So that jobs will be triggered based on priority.

3 Likes

Agree + maybe manual queue item priority?

That would be an awesome feature :smiley: !

Hi,

Any update on the availability of this feature?
Thanks.

it’s comming in 19.10!

10 Likes

Awesome! Just been discussing how some robots to start based on queue items with the team. When this due to be released?

Hi @badita,
Coming back to the original question, what would be the best solution for this question given the features we have till date.
“can a bot be initiated from the entry of an item into a queue or does the bot need to be running constantly looking for a queue item?”

@ankit002 Looks like its a “Scheduler” item. They’ve renamed it Triggers. Now a trigger can be a time OR a new item is added to the queue.

If you read that description, it reads “Start a job when new items are added to the queue” - thus initiating a bot from the entry of an item into a queue.

The trigger takes the place of needing a bot monitoring a queue.

1 Like

Thanks for your response @sagacity.

So i found Trigger under schedule section in Orchestrator like you mentioned. There are options to set the frequency for the schedule to run, How do i set it to Start a job automatically when a new item is added to queue?

@ankit002 It hasn’t been released yet. Its planned for version 19.10.