Multiple Process Queue Management

[lakshay.verma]

Hi forum Members,

Greetings of the day.
I have 4 processes built on re-framework (Queues not utilized previously as they don’t have the orchestrator). Now we plan to introduce a queue mechanism in it. These processes are dependent on each other and run in sequence 1–>2–>3–>4. and run after each other except for process 4 which runs after a week and process 1,2,3 runs multiple times a day but always in sequence.

1. As far as I can understand the best way to move in this scenario is to use a unique queue for each process and a queue item will be added on successful completion of the previous process. On failure of the previous process, no queue items should be added.

2. Process 4 needs to check the status and might need to run multiple times to make sure the correct status is updated, in this scenario new queue item will be in progress and can not be utilized on next run, I think I can use set transaction progress and would be the best practice.

I love to see responses from the leaders and share their experiences, and best practices to be followed in this scenario. Let me know if you need more information.

[Anil_G (Anil Gorthi)]

@lakshay.verma

1. Only for starting the next bot you want to add queue items? If so rather than that we can use start job api or start job activity to start the next job after successful completion of the precious job

2. And the queue item for the final process …you can postpone the item so that the same queue item will be in new state again…instead of using the progress status…and thai depends on for how long the item needs to be in in progress and will multiple items be there in in progress…if so going with postpone transaction item is better

And for maintainance we can use separate…but for ease if we want to use a single queue by having different references we can use a single queue as well

Hope this helps

Cheers

[lakshay.verma]

@Anil_G

No, The idea is not start the job. That is already being handled.
Idea is to provide the mechanism to run job individually. Let’s assume

The process can run only in this way.

1. Generate some request number from a third party website for all the records that are part of the excel file (100 records in each run). Assuming successful run is 80 and 20 failed
2. For those 80 records, I extract some more info from a local db… assuming I am able to get 60 successful records and 20 failed
3. Now for those 60 records submit application on a third party website. 40 successful records and 20 failed

Now assuming above scenario, bot needs to run all records for process 1, after process 2 and after process 3

And one week later for process 4.
All these are independent process but can only run in sequence.

Like I said before starting a bot is not an issue, What I need to know is the best practice that can be followed

My current approach is to provide separate queue to each process and for process 4 keep updating the transaction progress

[Anil_G (Anil Gorthi)]

@lakshay.verma

Fors the first 3 adding queue items makes sense…as you have transactions and can setup queue based triggers so that immediately after adding the bots can start processing

Now for fourth process …if you are leveraging update transaction then it wpuld work only in the acenario where you will have the item be in in progress state…so if you want to process the item in one go then you can have one more item and process in one go…or as specified other approach would be to use postpone queue item if you want to check again

Cheers