Duplicate Jobs - Triggering The Same Process Twice With Time Trigger Set At The Same Time

How to resolve the duplicate jobs for the same time trigger?

This could happen due to some zombie triggers in the quartz tables, which could cause double execution of the jobs.

Resolution: There are 2 ways of fixing that as following

Note: Ensure to take backup of the Database before performing the below

Approach A:: Zombie delete only method:

  1. From Orch gui disable all triggers (you have a check all checkbox and disable). This will delete the correct triggers from quartz table and this is normal, as we want to isolate the zombie entries only.
  2. Inspect and run the following script (note that first part is only for trial or improvement of knowledge in quartz table relationship, second part will flush zombie triggers, be it time or queue ones)

-- Quartz - Jobs relationship

select * from [quartz].[QRTZ_TRIGGERS] QT

join [quartz].[QRTZ_CRON_TRIGGERS] QCT on QT.TRIGGER_NAME = QCT.TRIGGER_NAME

join [quartz].[QRTZ_JOB_DETAILS] QJD on QT.JOB_NAME = QJD.JOB_NAME

join [dbo].[ProcessSchedules] PS on PS.ExternalJobKey = QT.JOB_NAME;

select * from [quartz].[QRTZ_TRIGGERS] QT join [dbo].[ProcessSchedules] PS on PS.ExternalJobKey = QT.JOB_NAME;

select * from [quartz].[QRTZ_TRIGGERS] QT join [quartz].[QRTZ_CRON_TRIGGERS] QCT on QT.TRIGGER_NAME = QCT.TRIGGER_NAME;

select * from [quartz].[QRTZ_TRIGGERS] QT join [quartz].[QRTZ_JOB_DETAILS] QJD on QT.JOB_NAME = QJD.JOB_NAME;

  • Flushing zombie triggers which exists in quartz tables after all custom triggers are disable from Orchestrator site.

Note: Disable all triggers from Orchestrator GUI before executing below commands

update [dbo].[ProcessSchedules] set Enabled = 0 where ExternalJobKey in (

select QT.JOB_NAME from [quartz].[QRTZ_TRIGGERS] QT

join [dbo].[ProcessSchedules] PS on PS.ExternalJobKey = QT.JOB_NAME

);

delete from [quartz].[QRTZ_CRON_TRIGGERS]

where TRIGGER_NAME in (select QT.TRIGGER_NAME from [quartz].[QRTZ_TRIGGERS] QT

join [quartz].[QRTZ_CRON_TRIGGERS] QCT on QT.TRIGGER_NAME = QCT.TRIGGER_NAME where QT.TRIGGER_GROUP <> 'ORCHESTRATOR_INTERNAL');

delete from [quartz].[QRTZ_JOB_DETAILS]

where JOB_NAME in (select QT.JOB_NAME from [quartz].[QRTZ_TRIGGERS] QT

join [quartz].[QRTZ_CRON_TRIGGERS] QCT on QT.TRIGGER_NAME = QCT.TRIGGER_NAME where QT.TRIGGER_GROUP <> 'ORCHESTRATOR_INTERNAL');

delete from [quartz].[QRTZ_TRIGGERS] where TRIGGER_GROUP <> 'ORCHESTRATOR_INTERNAL';

  1. Post above commands execution, re-enable triggers from Orchestrator site, no restart needed.

Approach B:: Flush all Quartz tables

  1. Same as Approach A, step 1 - From Orchestrator GUI( graphical User Interface) disable all triggers (you have a check all checkbox and disable). This will delete the correct triggers from quartz table and this is the normal expected behavior, as isolation of only zombie entries is desired.
  2. Stop Orchestrator server from IIS
  3. In UiPath.Orchestrator.dll.config file search for <add key="Database.EnableAutomaticMigrations" value="false" /> and switch it to true (case sensitive)
  4. Run the following SQL command:

begin tran

delete quartz.QRTZ_SIMPLE_TRIGGERS

delete quartz.QRTZ_CRON_TRIGGERS

delete quartz.QRTZ_TRIGGERS

delete quartz.QRTZ_JOB_DETAILS

update dbo.ProcessSchedules set [Enabled] = 0 where IsDeleted = 0

commit

  1. Reset the IIS. To reset, goto orchestrator server -> Open CMD as admin and run the command IISRESET.
  2. Login and re-enable all triggers previously disabled.

Note : At next Orchestrator reboot, you need to switch the <add key="Database.EnableAutomaticMigrations" value="true" /> back to false.

It is recommended to use approach A, since it is less intrusive and also the second approach will flush all internal Orchestrator triggers.