So I’m working on a project in REFramework, using enterprise edition 2020.10.2. We are planning to run the final automation simultaneously on multiple machines (no VMs, just personal computers), and we will be using Excel queues (not Orchestrator) per project requirements.
Since this is my first time building a project of such scale, I want to make sure I am covering my bases thoroughly when it comes to designing the multi-bot architecture, which is what I am coming here for guidance on.
So far, I have built out a check so that the transaction items are tracked by status in the Excel queue, to prevent bots from duplicating work queue items (bots will only take the next transaction item if it is marked as “new” status).
Since the bot will be scraping data for each transaction item and writing the scraped data table into an excel report, I am planning to next build out a lock mechanism so that once a bot opens the Excel file, the other bots will have to wait and retry until the lock is released. (My lock will be a text file that the bot will make on opening the excel file and delete on closing the excel file)
Besides these two checks I covered above, are there any other considerations I should take into account while preparing the REFramework for such a multi-bot architecture using Excel queues?
I’ve read about the idea of having each robot write their results to individual files to prevent conflicts, and then combine those files into one at the very end. But I am not sure if this will be necessary given that I will be using the lock mechanism I described to prevent multiple bots from being in the same Excel file at once. I also want to design the project so that it can accommodate any number of bots, and, while I could be wrong, I am not sure that the effort to assign each bot their own output file will be worth it if I use the lock mechanism approach to the main output file.
What do you all think?