We have table in Sql server which will be having 10K records and time required for processing one item is 5 mins. In order to expediate the processing of records, we need to apply 3 bot for processing all records. How to design the multibot architecture for same.
As per my understanding, all robots must have read/write access to sql server. Sql table must have one column like status and by default value will be In progress. once record processed, status will be completed. Robot should take record which are in In progress status only. What other precaution need to be taken ?
I would think twice before using an SQL Table directly as a work queue. You can do it, but with such large number of cases, you should rather split up the process and use orchestrator queues. Why? Just so that you as developer or those maintaining your robot can trace each and every case.
One Dispatcher → Run by any one of the three robots → A Queue dedicated to the process or multiple queues exclusive to each robot (your choice) → Assets access (read / write)
Three Performing robots → Who take queue items and process them → On success they updated the source SQL Database with two columns a) IsProcessed b)DateTimeProcessed. Here it is recommended that your SQL table also has a unique indentifier per row so that you can update that specific row with a) IsProcessed b)DateTimeProcessed.
The three robots will work as a team and fetch items which have status “New” in the orchestrator queue. For example, the first bot starts and picks a new queue item and sets it to “In Progress” the second fetches the next queue item with status “New” and so on. In short, they wont cross paths with each other.
Boon and Curse of Modern Folders:
When you want to set up multiple robots in modern folders it is both easy and difficult at the same time. Easy because you can allocate three robot client machines to a single Machine Template. Diffcult because, you cannot a single Robot User. You need to ensure all three robots have a dedicated Robot UserName and Passwords or three unique Active Directory users in the organization. So there will be some communication with your IT support to get this setup correctly.
Hope this helps clear some of your doubts. Good luck!