Processing Items in Queues using multiple bots

Help
1

Hi,

I was trying to understand Queues better and had the following doubt. The GetTransactionStatus only gets transactions with ‘New’ Status. I was wondering how a second bot could access these items, after the first bot has read them and updated the status to ‘Successful’.

The purpose of queues is that multiple bots can use this data and update it into other processes. What am I missing?

2

Hi Alistair,

Once you have deployed your process to Orchestrator, you have the ability to choose which Bots to use to perform the job or you can dynamically assign bots which means that Orhcestrator will use any available bot to process items in the queue.

GetTransactionStatus will only fetch items with “New” status - correct.
See: Queue Item Statuses

Multiple bots can operate or pick up transactions from a queue but a single bot will pick up a transaction and will process this transaction through to the end. The status of the transaction will then be updated depending on whether it was a success or failure for that single transaction. Then the Bot will move on to the next transaction in the queue.

The idea of using multiple Bots allows more robots to pick up items in the queue at the same time. This would reduce the overall time it would take to clear the queue as you would have more than 1 Bot working the queue at the same time. Each bot will handle a single transaction in this case.

From your question it seems as though you want 1 process to be picked up by multiple robots. It is possible to do it this way but you would need to rework your Project. i.e. you would need to create a new Reframework process to Load the Queue, another Reframework process to Get Items in Queue and another Reframework Process to Process the transaction. Having these processes split in to 3 will allow you to configure Orchestrator to use Bot1 to Load Queue, Bot 2 to Get Transactions and Bot 3 to Process. For me, it depends on the number of transactions you need to get through.

I hope this helps,

Sanjay

4 Likes
3

Sanjay, Thanks for your detailed response. I get a much clearer picture of how this functions now. Just had a doubt wrt to something you wrote.

Quoting your first line

“Multiple bots can operate or pick up transactions from a queue but a single bot will pick up a transaction and will process this transaction through to the end.”

Can you explain the difference between ‘can operate or pick up transactions from a queue’ and ‘will pick up a transaction and will process this transaction through to the end’

I thought once you picked up an item from a queue (using get transaction item), you would have to process it through the end and update its status. There is no way of just picking up a transaction without affecting its status.

5

You cannot process the Completed transaction. The solution for this to create new queue as “Complete Queue” and push to this queuee. BOT 3 will check the “Complete Queue” and perform the job.

6

Hi Alistair,

I think you got the wrong way of understanding about multi-bot process. Let me explain you clearly.

Multi-bots are used to process a queue items on multiple bots(multiple machines). For say, you have an excel with 10000 records to process, and each record takes around 2 mins of time to complete the work, then it will take 10000*2=20000 min= 333hours which is very long duration. So here comes the multi bot concept of running this queue on multiple machines(each robot is connected to a machine). If you have 10 bots on hand to execute this process, all the bots share the same process to execute on their own machine. Remember, each time before picking up the record, it will pick the ‘New’ transaction in the queue. If one bot is already processing the queue then its status is ‘In progress’, so another bot wont pick this transaction, it will pick either new/failed transaction as per the design.

I think this explains your doubt.

Regards,
Sravya V

7 Likes
8

Good explanation…cleared my doubt

9

Very clear explanation, although would like to know to load items in Queue is done by another bot right?

10

Hi, Nice explanation. I have 3 unattended bot which need to run on a file having 100000 records in parallel. Is there any step by step reference available?

11

we are running many business process on multi-bot configurations, where one script is assigned to one bot to load items in a queue. A reframework process is used by many bots to pick up queue items which can be processed concurrently by multiple bots, that is not a problem as orchestrator & queue logic manage this for us.

Now, we want to build a business process that manages better system exceptions including system unavailability. I’m currently looking at building a process that leverages transactions. So one process would pick up a queue item, start a transaction by adding it to a seperate, transaction queue. Then another reframework process running on multiple bots would pull in a transaction item (filter: status, reference etc.) the problem is that if one bot, does a Get Transaction Items with a filter, the same process, running on a different bot would do the same Get Transaction Items with the same filter, and each running simultaneously would pick up the same list of transaction items from the transaction queue, if the script updates the status of say the first item in the list, the second bot would likely also be processing the same first item from the list. So from my understanding, both bots would be processing the same queue item at the same time until one updates the queue item status, in which then subsequent thread of the process would return a different list of queue items, how can we design and implement a transaction where a business process is executing a long running process, accessing say 2-3 applications, which may/may not be available etc.

12

If you use the RE framework to consume from a queue, orchestrator will only allow one bot to consume a particular queue item/transaction.

There is no need to over-engineer a solution with an additional queue.

You can design a business process automation in which you use an inbound queue, and an outbound queue to allow multi-bot arrangement. The outbound queue is required if you say, generate an XML file that contains all records processed in the queue. If multiple bots are running, you cannot expect that each one will be able to read/write-append to the same file say on a network accessible drive.

An outbound queue in this case make sense, so that each bot writes information to a queue, and have a seperate RE framework process to consume all queue items and put them/append them in the file.

13

The purpose is not to have two different jobs process the same item. The purpose is to have multiple jobs NOT try to process the same item. If you need the same item processed by two different jobs, add it to two different queues.

14

How is that not an additional queue?

15

that additional queue - outbound is only required depending on your functional requirements… what is your scenario, perhaps we can help? A queue is used when you need to control execution of transactions, so that bots cannot consume the same transaction or queue item concurrently.

the set transaction status is used to set the status of the queue item to Failed, or Success.