Issue in Orchestrator version 2019.10.15

Hi,

We have upgraded Orchestrator to 2019.10.15 version and facing some issue in this.

  1. It shows running and remaining queue items 0 while there be New queue items in a particular queue.
  2. it shows incorrect count of Queue items.
  3. Dashboard also shows wrong robot which is unresponsive.

Please resolve this problem as it is difficult for us to monitor the queue item in a queue.

Regards,
Satyendra

Hi @satchauhan
I actually didnt experience your things yet :stuck_out_tongue_winking_eye:

But will give you few suggestions
Go to your IIS and restart the orchestrator
Delete pre created robot and machine and re produce

we have already done it. there is a problem in orchestrator, it is showing wrong details. please look into this.

Hello we are facing the same issues.

We found that some SQL processes in Orchestrator MS SQL database are sleeping and locking tables/rows.

We have temporary fixed problem by manually killing blocking process, but the issue reappeared.

The first time we have found UPDATE statement which was blocking other processes
(@state nvarchar(6),@jobName nvarchar(26),@jobGroup nvarchar(21),@oldState nvarchar(14))UPDATE quartz.QRTZ_TRIGGERS SET TRIGGER_STATE = @state WHERE SCHED_NAME = 'UiPathQuartzSchedulerSystem' AND JOB_NAME = @jobName AND JOB_GROUP = @jobGroup AND TRIGGER_STATE = @oldState

Second time we found this statement
(@lockName nvarchar(14))SELECT * FROM quartz.QRTZ_LOCKS WITH (UPDLOCK,ROWLOCK) WHERE SCHED_NAME = 'UiPathQuartzSchedulerSystem' AND LOCK_NAME = @lockName

Any ideas why it might happen?

1 Like

Yes there is some issue in this version of orchestrator. Deleting robots and machine can not be the solution as they are doing their job fine.
Previous version of orchestrator refresh everything automatically in a interval but now we have to click on refresh button every time.
not showing queue item in a queue is a big issue which should be solve on urgent basis instead of giving unnecessary work around.

Please solve the problem as soon as possible.

Regards,
Satyendra

Is there any solution of this as Queues are still not showing number of items. we have restarted IIS, created robot again but no success, it is still not updating

Hi @satchauhan
Did you manage to find a solution for this issue? We are facing the same issue in Orchestrator 2019.10.15.
Best Regards,
Martin

Not on 2019.10 yet, but this could be one of the reasons

Caution!

Prior to upgrading to v2019, make sure that you are not using the same MachineKey on different hosts when connecting your unattended robots. Due to changes in the robot heartbeat, this can result in connectivity issues.

The issue may be related with blocked/error triggers in [quartz].[QRTZ_TRIGGERS] table. Switching the trigger to “Waiting” state usually fix problems. Also custom triggers can be switched on/off from Orchestrator, which will force re-calculation of ticks according with cron expression. Locking tables are normal when there are transactions to be done.

Hi, I think we are having exactly this issue:


What is the proper way to fix it, i.e. change to WAITING status or similar?

A proper way to fix it is to check your schedulers and ask your dba what is causing the block status and act accordingly. As workarounds you can set trigger state to Waiting or you can try to flush all quartz tables while having the site stopped from IIS and all schedulers disabled.

Thanks for the reply. Could I get some more details about the suggestions? :slight_smile: Fox example, “check your schedulers”? What to check precicesly and where? I’m afraid I don’t know how email alerts schedules can be checked (only that the interval can be set in web.config)? How would I set trigger state to “Waiting”? By changing the value in the database? How do I “flush quartz tables” - does this refer to some specific sql command? We have usually “fixed” this by restarting IIS, it usually clears the situation. But we would of course like to avoid any downtime as this is only about the email alerts.

In the Orch site check on Automation>Trigger page, every custom trigger for a scheduled job will be present with a trigger_group default, so if you disable them from site, they will not be present in the [QRTZ_TRIGGERS] table and only internal ones will be shown, so you can flush the default ones (will not share steps here). Your mail alerts is set somewhere else, specifically in the [quartz].[QRTZ_CRON_TRIGGERS] table using cron expressions.
If you need to modify the state of a trigger simply update the sql table and set value to ‘Waiting’, that should do it. ( update [quartz].[QRTZ_TRIGGERS] set [TRIGGER_STATE] = ‘WAITING’ where [TRIGGER_NAME] = ‘NotificationDistributerJob’ )
For flushing quartz table please open a ticket with the support team, as the procedure will usually include a db backup (or at least some table backups) and some extra steps.