How to modify the REFramework to read the data from more queues?

[Andreea_Nistor]

Hello, I have a project where I have to read 8 queues names from orchestrator, one by one, then for each Queue Name, process the items within, then move to the next Queue name. How can i do that with REFramework? Where should I initiate the list of Queues that i am interested in? and how do i do the loop ? Please help

[robin1 (Robin)]

Is the process being run on the Queue Data the same for every queue, whats the need to have 8 queues?

[Jon_Smith]

Wouldnt the architecture be more robust if these 8 queues were each executed independently from different processes? What is your need/use case for wanting them to be combined in a single process instance?

[ashokkarale]

@Andreea_Nistor,

Best practice would be utilizing REFramework provided input argument

in_OrchestratorQueueName

Just create 8 triggers with in_OrchestratorQueueName 8 unique queue names and let them run independently.

Modifying REFramework would be overcomplimenting the simple architecture.

[Andreea_Nistor]

The need for 8 queue is that each one represents an application and contains different information, that need to be processed separately to send emails at the end of the process for each App.

[Jon_Smith]

I’d disagree, best practice would be to separate this into 8 processes and not make a single process overly complex.

[jeevith (Jeevith Hegde, Ph.D.)]

@Jon_Smith

Wouldnt the architecture be more robust if these 8 queues were each executed independently from different processe

We do this for all our processes. Helps us a lot when things go wrong. If things go bad only one process fails and rest of them can still run.

We utilize the in_OrchestratorQueueName in the reframework to allocate the queuename statically for each trigger of a dispatcher / performer process.

@Andreea_Nistor This will also mean that you have dedicated logs (via Jobs) and triggers, which will be a god sent when you have to troubleshoot in production.
This also allows you to schedule your runs much more efficiently than dasiy chaining processes with in single for each queue within a process.

flowchart LR

DispatcherTrigger_Q1 -->|InputAgrument=Q1| DispatcherProcess
DispatcherTrigger_Q2 -->|InputAgrument=Q2| DispatcherProcess
DispatcherTrigger_Q3 -->|InputAgrument=Q2| DispatcherProcess

DispatcherProcess -->|Populates|Q1
DispatcherProcess -->|Populates|Q2
DispatcherProcess -->|Populates|Q3

Similarly for the performer

flowchart LR

PerformerTrigger_Q1 -->|InputAgrument=Q1| PerformerProcess
PerformerTrigger_Q2 -->|InputAgrument=Q2| PerformerProcess
PerformerTrigger_Q3 -->|InputAgrument=Q2| PerformerProcess

Choice of trigger is use case specific.

1. Dispatcher is set to time schedule trigger and performer is set to queue triggers.
2. Dispatcher is set to time schedule trigger and performer is set to time triggers.

[Andreea_Nistor]

Where is that screen from ?

[Andreea_Nistor]

Exactly what i need to implement, my Dispatcher populates the queues at a specific time, then the Performer has to read from each queue the items and process them. But i don’t know how to create, and where to add a loop of queues, in REF workflows.

[ashokkarale]

It’s from REFramework Main.xaml which will reflect as arguments while creating process or trigger configuring @Andreea_Nistor

[Andreea_Nistor]

I don’t have an argument named in_Process in Mail.xaml or any other workflow…

[jeevith (Jeevith Hegde, Ph.D.)]

All you have to set up are triggers for your Dispatchers and Performers (REFramework)
It comes with an in_OrchestratorName input argument in main.xaml.

In your relevant folder go to Automations → Triggers

Set up one trigger at a time, the UI will walk you through.

This is the place where you will set your queue name as input argument in you triggers. You will see all your arguments of your process

[ashokkarale]

@Andreea_Nistor,

That’s custom one. You focus on this one

This is the way

[Andreea_Nistor]

So, how can I pretty ask my robot to check for more queue names here? Can i use Config file and make a list of the Queues?

[ashokkarale]

@Andreea_Nistor,

In Initialization State after InitAllSettings invoke, REF checks which queue to use. If the queue name is provided through argument, that will be preferred, else queue name from Config will be considered.

[Andreea_Nistor]

ok, this is not a bad idea, but then in config file, what name should i put for the Queue?

[Andreea_Nistor]

i understand that, and how to loop thru multiple names? when in Get transaction data is asking for a queue name ?

[jeevith (Jeevith Hegde, Ph.D.)]

You are essentially not going to use config file to get your queue names. Let the process fetch it from the triggers directly.

When you develop, you can set a test queue and when you go to production, you set the queue name seperately for each trigger. You do this only once or whenever you want to add more queues .

In short, 8 queues = 8 dispatcher triggers + 8 performer triggers. I know it is a manual work (as triggers do not support cloning) but it is set it and forget.

There is no need to loop at all. You are not going to use config or another logic in your REFramework process. Let the trigger be the guide.

[ashokkarale]

@Andreea_Nistor,

Create 8 triggers and every trigger will have your unique queue names being passed as argument.

So this Get Transaction Item will get the queue name from arguments.

[Andreea_Nistor]

Sounds like an idea i want to try , but one question: My Dispatcher is adding the items in those 8 queue, because it performs 8 app checks, the trigger will be a time trigger, once per 30 min 24/7. Can your solution work? With 1 trigger for the Dispatcher and 8 for the performer.